Coding

[vh]

not gonna do that because nothing cool is going to come out. the picture i posted was a combination of careful hand tuning and a lot of luck.

[vh]

today in "really weird python functions":

>>> 'hello world'.title()
'Hello World'


come on guys was that really a necessary language feature?

[Bla]

I still don't get how I didn't mess up several times when typing in the second order dt terms and it didn't mess up my simulation to give e^308 photons. Now time to write the second order terms that include the extra laser pulse

Nop I haven't efficientized the math yet. I'm not even sure it's gonna be more efficient with the 2nd order terms than just making the time step lower and run the same amount of time. Obviously slows down the simulation a lot

[vh]

o man dat subexpression elimination waiting to happen

[atomic7732]

script to generate false color GOES-16 imagery



(DATETIME will need to be modified cause the server i use only keeps like the past 24 hours of data)

[FiahOwl]

This message is only viewable with Universe Sandbox Galaxy Edition. Access it and much more with promo-code '181002'.

[atomic7732]

have some messy code that does this

[atomic7732]

waves and cyclones now have intensities

and waves have to form into cyclones



oh yeah, and there's land now

requires numpy, matplotlib, netCDF4, and this dataset from here

[FiahOwl]

This message is only viewable with Universe Sandbox Galaxy Edition. Access it and much more with promo-code '181091'.

[atomic7732]

uh i'm not sure if i'll make a pavala version in full, but soleani cyclone center will feature the amalthean region at least

edit: some updates





the second one has a feature where if a cyclone is moving south quickly it weakens... cause it should

and fun fact, 15 epac storms with 5 atl storms in the months of july and august seems weird right but last year it was 13 and 4 (and the sim used last year's sst's so...)

[Darvince]

goond

[atomic7732]

new update moves the ridges and troughs based on month so now you can run full seasons (and even years, although i haven't tested full years yet)

i did this 6 month run from june to november of 2012 and got

	sim	irl
EPAC	17-8-2	17-10-5
ATL	19-4-1	19-10-2

where in X-Y-Z
X is the number of tropical storms
Y is the number of hurricanes
Z is the number of major hurricanes (category 3 or higher)

so i am legit impressed at my own creation

also the color scheme i'm using:
white - tropical depression
yellow - tropical storm
yellow-orange - severe tropical storm
orange - hurricane (cat 1 or 2)
red - major hurricane (cat 3 or 4)
purple - cat 5 hurricane

[atomic7732]

[07:54]   Kalassak   WEW
[07:54]   Kalassak   it runs so much quicker now
[07:54]   Kalassak   BLESS SYULE GOD FROM THE HEAVENS



a two month sim like that used to take 9 minutes now it takes like 2.5 or less

and the pressure field calculations went from like 1 second to .001 seconds so that's great, now i just have to optimize the graphics time

[vh]

recorded in 1080p60 cause i can

https://www.youtube.com/watch?v=1ZbHdFGfQZk

watch a computer figure out how to swing this heavy pendulum into a vertical position.

the radial arrow indicates the direction and magnitude of torque applied.

this is super interesting: the algorithm decides it can't apply enough force to swing the pendulum into vertical position in one go, so instead, it pushes the pendulum up most of the way in one direction, then lets it fall and reach the top from the other direction!

none of this strategy is hardcoded of course -- the algorithm "discovers" it. the only thing the algorithm is given is a sandbox/simulator pendulum, where it can do arbitrary things to the pendulum and see what happens.

[vh]

no code this time but here's the architecture of the code which runs blacraft. it probably totals to between 500 and 2000 loc.
edit: actually 642 nonblank lines, as computed by this great bash command i just wrote:

Code: [Select]

find -L . | grep py$ | xargs cat | sed '/^\s*$/d' | wc -l
blue boxes are external (such as irc, amazon web services, and blacraft's cloud server)
yellow boxes are pieces of code which sit on my computer

the right side is pretty simple -- we have the irc interface which parses and validates all irc lines and sends commands / forwards messages from the UBV box. the irc interface should also reconnect in case of time out or related issues, but that part of the code is untested. ubv handles the logic of handling irc commands, which right now is pretty simple -- it just passes them on to the coordinator on the top

on the left side, we have the aws interface, which talks to both the server and also aws. it does so through a bunch of different ways -- ssh connections, directly pinging the spigot server, and making requests to AWS to start up and shut down the machine. the aws interface provides all of the primitives such as acquiring a machine, shutting down a machine, or running the spigot jar on the server.

on top of that, we have the server controller, which assembles these primitives into a small number of commands, such as server startup, server shutdown, query population, etc. it has to take care of stuff such as : the ssh server on the blacraft machine doesn't boot up instantly, so even after the machine is up, you need to wait the right about of time. it also has to robustly handle errors and timing issues, so spamming server on and server off won't cause the interface to go haywire (yes the aws interface is rather fragile). huge potential for race conditions in this box that i had to be careful about

then, on top of that, we have the coordinator, which talks to both UBV and the server controller. the coordinator has the job of passing commands from ubv to the server controller, as well as passing messages (such as the success of the startup command), back over to UBV so we on irc will be able to know when the server is up. it uh also doubles as the main object which initializes everything, both the server controller and also ubv, which may or may not be bad design practice.

so yeah there you have it

[Bla]

Impressive

[vh]

figured out how to put text on images today

[vh]

second image: a picture of how the chair looks
first image: how the chair looks when melted (an attempted 3d reconstruction of the chair)
third image: how the chair is supposed to look from all those angles

basically i'm computing a 3d model of a chair based off of one picture
using brainz i mean ai i mean neural nitwerks

[Bla]

Note bade

[Darvince]

have you tried increasing the layers of neurons

[vh]

sYnThEsIzEd ChAiR

yeah this needs improving

[vh]

1. input image
2. sYnThEsIzEd ChAiR (yeah it's missing the legs)

[vh]

1. input image
2. generated 3d model

[vh]

i wrote a nifty utility function which basically takes a container object (list, dictionary, or tuple), and strips all the actual content out of it, replacing all the values with None

Code: [Select]

def iscontainer(obj):
    return isinstance(obj, list) or isinstance(obj, dict) or isinstance(obj, tuple)
   
def strip_container(container):
    assert iscontainer(container), 'not a container'
   
    if isinstance(container, list) or isinstance(container, tuple):
        return [(strip_container(obj) if iscontainer(obj) else None)
                for obj in container]

    else:
        return {k: (strip_container(v) if iscontainer(v) else None)
                for (k,v) in container.items()}

so if you have dictionaries and lists and nested dictionaries and lists containing a bunch of big strings and numbers and class instances, this replaces all those bottom level entries with None values, which makes it really easy to read the "outline" of a complicated "data structure"

for example, here's what it looks like when i apply it to some code i'm debugging:

Code: [Select]

{'gt': {'depths': None,
        'masks': None,
        'pose': None,
        'view': None,
        'views': None},
 'pred': {'flow': {'pred_flow': None, 'pred_view': None},
          'par': {'preds': {'depth': None,
                            'dx': None,
                            'dy': None,
                            'mask': None,
                            'view': None},
                  'reproj': {'body': {'depth': None,
                                      'mask': None,
                                      'view': None},
                             'view': {'depth': None,
                                      'mask': None,
                                      'view': None}}}}}

if i hadn't used this function, all of those None values would've printed with huge numpy arrays which would've made it impossible to read

[vh]

a mysterious memory leak -- a very slow one too, about 1gb per hour

vertical: memory used in GB
horizontal: iteration number

[Bla]

Mysterious. Imo it helps to close the window, when it's windy the memory tends to blow out

[vh]

this plot of a stripped down version of the code suggests that maybe the memory leak levels out after a few GB and stops

[vh]

the following images are plots of how many milliseconds are used to process each chunk. in order to run at 20 tps, the server cannot spend more than 50ms on all currently loaded chunks.

the following are maps of 100x100 chunk areas, or 800m by 800m
leftmost: synthetically generated lag map by me. brighter means more ms used to process chunk
rightmost: naive estimates of the chunk lag by simply diving 1000.0 by the tps
middle: my algorithm for estimating the chunk lag

things the algo can deal with
 - the fact that tps is averaged over the last minute, during which you could've moved a lot
 - multiple players simultaneously loading many chunks
 - in the first image, there is a single chunk which takes 100ms to process. in the second image, there are 20 chunks, each which has 40ms of processing time. note that by itself, the server could still operate at 20tps -- so it's only when two of these chunks are loaded that the tps lag happens. yet, my algorithm can still locate which chunks caused the lag

actually the multiple player support is in theory -- haven't implemented it yet. but don't fear. i'm about to implement it.

[vh]

so here's a demonstration of multi-player lag computations

so first look at the right most plot. there are two horizontal lines -- one top on bottom, which shows the paths taken by the two players.

for about 1h of irl time, the top player moves right across the map, while the bottom player stays in the bottom left corner.

then in the next 1h of irl time, the top player moves left across the map, while the bottom player moves right. they do this at the same speed so that they cross in the middle

as shown by the leftmost ground truth lagmap, there is only one laggy chunk, which is in the middle of the top player's path. however, the naive "observation based" strategy would get confused and think that the bottom player is also responsible for the lag, due low tps observed when the bottom player is in the middle.

on the other hand, my algo pretty much pinpoints the lag just in the top player's path. note that there's a lot of vertical spread to the prediction, since both players are moving horizontally, and it's impossible to tell which chunk out of the 21 chunks in the vertical column was the lag causer without some vertical movement.

[vh]

wrote a long rambly thing about my OO approach to DL