Welcome, Guest

Author Topic: Coding  (Read 125553 times)

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #660 on: May 08, 2017, 01:15:35 PM »
not gonna do that because nothing cool is going to come out. the picture i posted was a combination of careful hand tuning and a lot of luck.

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #661 on: May 14, 2017, 02:59:40 AM »
today in "really weird python functions":

>>> 'hello world'.title()
'Hello World'


come on guys was that really a necessary language feature?

Bla

  • Global Moderator
  • *****
  • Posts: 1011
  • The stars died so you can live.
Re: Coding
« Reply #662 on: May 30, 2017, 11:50:13 AM »
I still don't get how I didn't mess up several times when typing in the second order dt terms and it didn't mess up my simulation to give e^308 photons. Now time to write the second order terms that include the extra laser pulse

Nop I haven't efficientized the math yet. I'm not even sure it's gonna be more efficient with the 2nd order terms than just making the time step lower and run the same amount of time. Obviously slows down the simulation a lot


vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #663 on: May 30, 2017, 12:25:55 PM »
o man dat subexpression elimination waiting to happen

atomic7732

  • Global Moderator
  • *****
  • Posts: 3753
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #664 on: June 04, 2017, 03:42:38 AM »
script to generate false color GOES-16 imagery



(DATETIME will need to be modified cause the server i use only keeps like the past 24 hours of data)

FiahOwl

  • *****
  • Posts: 1234
  • This is, to give a dog and in recompense desire my dog again.
Re: Coding
« Reply #665 on: June 04, 2017, 11:27:10 PM »

atomic7732

  • Global Moderator
  • *****
  • Posts: 3753
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #666 on: June 13, 2017, 12:46:52 AM »
have some messy code that does this



« Last Edit: June 13, 2017, 02:04:25 AM by atomic7732 »

atomic7732

  • Global Moderator
  • *****
  • Posts: 3753
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #667 on: June 14, 2017, 03:55:22 AM »
waves and cyclones now have intensities

and waves have to form into cyclones



oh yeah, and there's land now

requires numpy, matplotlib, netCDF4, and this dataset from here

FiahOwl

  • *****
  • Posts: 1234
  • This is, to give a dog and in recompense desire my dog again.
Re: Coding
« Reply #668 on: June 15, 2017, 01:15:36 AM »
can't wait for the pavala version tbh, when's that coming

atomic7732

  • Global Moderator
  • *****
  • Posts: 3753
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #669 on: June 15, 2017, 01:16:48 AM »
uh i'm not sure if i'll make a pavala version in full, but soleani cyclone center will feature the amalthean region at least

edit: some updates





the second one has a feature where if a cyclone is moving south quickly it weakens... cause it should

and fun fact, 15 epac storms with 5 atl storms in the months of july and august seems weird right but last year it was 13 and 4 (and the sim used last year's sst's so...)
« Last Edit: June 15, 2017, 03:17:58 AM by atomic7732 »

Darvince

  • *****
  • Posts: 1811
  • gab
Re: Coding
« Reply #670 on: June 15, 2017, 08:14:54 AM »
goond

atomic7732

  • Global Moderator
  • *****
  • Posts: 3753
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #671 on: June 19, 2017, 02:20:05 AM »
new update moves the ridges and troughs based on month so now you can run full seasons (and even years, although i haven't tested full years yet)

i did this 6 month run from june to november of 2012 and got



simirl
EPAC17-8-217-10-5
ATL19-4-119-10-2

where in X-Y-Z
X is the number of tropical storms
Y is the number of hurricanes
Z is the number of major hurricanes (category 3 or higher)

so i am legit impressed at my own creation

also the color scheme i'm using:
white - tropical depression
yellow - tropical storm
yellow-orange - severe tropical storm
orange - hurricane (cat 1 or 2)
red - major hurricane (cat 3 or 4)
purple - cat 5 hurricane
« Last Edit: June 19, 2017, 03:40:03 AM by atomic7732 »

atomic7732

  • Global Moderator
  • *****
  • Posts: 3753
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #672 on: June 22, 2017, 03:14:55 AM »
[07:54]   Kalassak   WEW
[07:54]   Kalassak   it runs so much quicker now
[07:54]   Kalassak   BLESS SYULE GOD FROM THE HEAVENS



a two month sim like that used to take 9 minutes now it takes like 2.5 or less

and the pressure field calculations went from like 1 second to .001 seconds so that's great, now i just have to optimize the graphics time

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #673 on: September 30, 2017, 12:08:58 AM »
recorded in 1080p60 cause i can



watch a computer figure out how to swing this heavy pendulum into a vertical position.

the radial arrow indicates the direction and magnitude of torque applied.

this is super interesting: the algorithm decides it can't apply enough force to swing the pendulum into vertical position in one go, so instead, it pushes the pendulum up most of the way in one direction, then lets it fall and reach the top from the other direction!

none of this strategy is hardcoded of course -- the algorithm "discovers" it. the only thing the algorithm is given is a sandbox/simulator pendulum, where it can do arbitrary things to the pendulum and see what happens.

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #674 on: October 05, 2017, 11:50:27 PM »


no code this time but here's the architecture of the code which runs blacraft. it probably totals to between 500 and 2000 loc.
edit: actually 642 nonblank lines, as computed by this great bash command i just wrote:
Code: [Select]
find -L . | grep py$ | xargs cat | sed '/^\s*$/d' | wc -l
blue boxes are external (such as irc, amazon web services, and blacraft's cloud server)
yellow boxes are pieces of code which sit on my computer

the right side is pretty simple -- we have the irc interface which parses and validates all irc lines and sends commands / forwards messages from the UBV box. the irc interface should also reconnect in case of time out or related issues, but that part of the code is untested. ubv handles the logic of handling irc commands, which right now is pretty simple -- it just passes them on to the coordinator on the top

on the left side, we have the aws interface, which talks to both the server and also aws. it does so through a bunch of different ways -- ssh connections, directly pinging the spigot server, and making requests to AWS to start up and shut down the machine. the aws interface provides all of the primitives such as acquiring a machine, shutting down a machine, or running the spigot jar on the server.

on top of that, we have the server controller, which assembles these primitives into a small number of commands, such as server startup, server shutdown, query population, etc. it has to take care of stuff such as : the ssh server on the blacraft machine doesn't boot up instantly, so even after the machine is up, you need to wait the right about of time. it also has to robustly handle errors and timing issues, so spamming server on and server off won't cause the interface to go haywire (yes the aws interface is rather fragile). huge potential for race conditions in this box that i had to be careful about

then, on top of that, we have the coordinator, which talks to both UBV and the server controller. the coordinator has the job of passing commands from ubv to the server controller, as well as passing messages (such as the success of the startup command), back over to UBV so we on irc will be able to know when the server is up. it uh also doubles as the main object which initializes everything, both the server controller and also ubv, which may or may not be bad design practice.

so yeah there you have it
« Last Edit: October 06, 2017, 12:10:10 AM by vh »

Bla

  • Global Moderator
  • *****
  • Posts: 1011
  • The stars died so you can live.
Re: Coding
« Reply #675 on: October 06, 2017, 04:01:41 PM »
Impressive

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #676 on: October 26, 2017, 10:46:02 PM »
figured out how to put text on images today


vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #677 on: October 27, 2017, 10:23:39 PM »
second image: a picture of how the chair looks
first image: how the chair looks when melted (an attempted 3d reconstruction of the chair)
third image: how the chair is supposed to look from all those angles

basically i'm computing a 3d model of a chair based off of one picture
using brainz i mean ai i mean neural nitwerks
« Last Edit: October 27, 2017, 10:30:01 PM by vh »

Bla

  • Global Moderator
  • *****
  • Posts: 1011
  • The stars died so you can live.
Re: Coding
« Reply #678 on: October 28, 2017, 05:27:32 AM »
Note bade

Darvince

  • *****
  • Posts: 1811
  • gab
Re: Coding
« Reply #679 on: October 28, 2017, 06:23:05 AM »
have you tried increasing the layers of neurons

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #680 on: October 28, 2017, 07:57:00 AM »
sYnThEsIzEd ChAiR

yeah this needs improving

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #681 on: October 28, 2017, 08:14:02 AM »
1. input image
2. sYnThEsIzEd ChAiR (yeah it's missing the legs)

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #682 on: October 29, 2017, 02:35:31 PM »
1. input image
2. generated 3d model

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #683 on: November 16, 2017, 07:48:11 PM »
i wrote a nifty utility function which basically takes a container object (list, dictionary, or tuple), and strips all the actual content out of it, replacing all the values with None

Code: [Select]
def iscontainer(obj):
    return isinstance(obj, list) or isinstance(obj, dict) or isinstance(obj, tuple)
   
def strip_container(container):
    assert iscontainer(container), 'not a container'
   
    if isinstance(container, list) or isinstance(container, tuple):
        return [(strip_container(obj) if iscontainer(obj) else None)
                for obj in container]

    else:
        return {k: (strip_container(v) if iscontainer(v) else None)
                for (k,v) in container.items()}

so if you have dictionaries and lists and nested dictionaries and lists containing a bunch of big strings and numbers and class instances, this replaces all those bottom level entries with None values, which makes it really easy to read the "outline" of a complicated "data structure"

for example, here's what it looks like when i apply it to some code i'm debugging:
Code: [Select]
{'gt': {'depths': None,
        'masks': None,
        'pose': None,
        'view': None,
        'views': None},
 'pred': {'flow': {'pred_flow': None, 'pred_view': None},
          'par': {'preds': {'depth': None,
                            'dx': None,
                            'dy': None,
                            'mask': None,
                            'view': None},
                  'reproj': {'body': {'depth': None,
                                      'mask': None,
                                      'view': None},
                             'view': {'depth': None,
                                      'mask': None,
                                      'view': None}}}}}

if i hadn't used this function, all of those None values would've printed with huge numpy arrays which would've made it impossible to read

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #684 on: January 12, 2018, 10:18:44 PM »
a mysterious memory leak -- a very slow one too, about 1gb per hour

vertical: memory used in GB
horizontal: iteration number

Bla

  • Global Moderator
  • *****
  • Posts: 1011
  • The stars died so you can live.
Re: Coding
« Reply #685 on: January 13, 2018, 05:10:55 AM »
Mysterious. Imo it helps to close the window, when it's windy the memory tends to blow out

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #686 on: January 15, 2018, 11:06:06 AM »
this plot of a stripped down version of the code suggests that maybe the memory leak levels out after a few GB and stops

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #687 on: January 15, 2018, 01:30:28 PM »
the following images are plots of how many milliseconds are used to process each chunk. in order to run at 20 tps, the server cannot spend more than 50ms on all currently loaded chunks.

the following are maps of 100x100 chunk areas, or 800m by 800m
leftmost: synthetically generated lag map by me. brighter means more ms used to process chunk
rightmost: naive estimates of the chunk lag by simply diving 1000.0 by the tps
middle: my algorithm for estimating the chunk lag

things the algo can deal with
 - the fact that tps is averaged over the last minute, during which you could've moved a lot
 - multiple players simultaneously loading many chunks
 - in the first image, there is a single chunk which takes 100ms to process. in the second image, there are 20 chunks, each which has 40ms of processing time. note that by itself, the server could still operate at 20tps -- so it's only when two of these chunks are loaded that the tps lag happens. yet, my algorithm can still locate which chunks caused the lag

actually the multiple player support is in theory -- haven't implemented it yet. but don't fear. i'm about to implement it.
« Last Edit: January 15, 2018, 01:35:42 PM by vh »

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #688 on: January 15, 2018, 02:02:19 PM »
so here's a demonstration of multi-player lag computations

so first look at the right most plot. there are two horizontal lines -- one top on bottom, which shows the paths taken by the two players.

for about 1h of irl time, the top player moves right across the map, while the bottom player stays in the bottom left corner.

then in the next 1h of irl time, the top player moves left across the map, while the bottom player moves right. they do this at the same speed so that they cross in the middle

as shown by the leftmost ground truth lagmap, there is only one laggy chunk, which is in the middle of the top player's path. however, the naive "observation based" strategy would get confused and think that the bottom player is also responsible for the lag, due low tps observed when the bottom player is in the middle.

on the other hand, my algo pretty much pinpoints the lag just in the top player's path. note that there's a lot of vertical spread to the prediction, since both players are moving horizontally, and it's impossible to tell which chunk out of the 21 chunks in the vertical column was the lag causer without some vertical movement.

vh

  • formerly mudkipz
  • *****
  • Posts: 1135
  • "giving heat meaning"
Re: Coding
« Reply #689 on: January 19, 2018, 07:51:53 PM »
wrote a long rambly thing about my OO approach to DL