Welcome, Guest

Author Topic: Coding  (Read 262418 times)

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #330 on: January 01, 2014, 06:12:28 PM »
kdf based on sha512

time complexity is O(n2)
memory complexity is O(n)

there are 3 functions in the code. hash is just sha512 as implemented by a library
algohash is the actual algorithm
algocollision is a generator object that yields keys with partial hash collisions -- similar to cryptocurrency mining

code:
Code: [Select]
import hashlib
import random

def hash(k,hasher):
    hasher.update(str(k))
    return int(hasher.hexdigest(),16)

def algohash(length,start):
    hasher = hashlib.sha512()
    plist = [start]
    for x in xrange(0,length):
        plist.append(hash(sum(plist),hasher))
    return hex(plist[-1])

def algocollision(length,bits):
    while 1:
        key = random.random()
        r = algohash(length,key)
        b = bin(int(r[2:-1],16))[2:].zfill(8)
        if b[1:bits+1] == '0'*bits:
            yield key

basically, there's a list. you put the input into the list.
then for n times, sum the list, hash the sum, append the result to the list

on a single thread, algohash takes about 3 seconds with a length parameter of 10000 and 12 seconds with a length parameter of 20000. with a length parameter of 100000, the expected time is 5 minutes

with a length parameter of 100000 the memory usage should be around 6.1 MB, making it extremely difficult to parallelize onto a gpu or asic. the average gpu runs from 5000 to 40000 concurrent threads. this would be 30 and 238 GB of ram respectively using the algohash algorithm
« Last Edit: January 01, 2014, 06:21:18 PM by vh »

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #331 on: January 03, 2014, 11:51:01 PM »
sloppy code but it works

Code: [Select]
import itertools
import pyglet
import random
from pyglet.gl import *
import noise
import numpy as np
import colorsys as cs
import math

win = pyglet.window.Window(resizable=True)
kb = pyglet.window.key.KeyStateHandler()
win.push_handlers(kb)
win.set_exclusive_mouse(True)

def randcol():
    return [int(random.random()*256) for x in range(3)]

def narray(x, y, s=200, p = 0.5, oct = 3):
    return noise.snoise2(x/s, y/s, octaves = oct, persistence = p)

def trimesh(vcmax, kmax, scale, offsetx, offsety, func, theight):
    num = vcmax*kmax
    const = (3**0.5)/2
    eo = [[[k*const*scale+offsetx,
            offsety+scale*(k/2.0+vc),
            theight*func(k*const*scale+offsetx, offsety+scale*(k/2.0+vc))
            ] for vc in xrange(0,vcmax)] for k in xrange(0,kmax)] #k, vc, 3list
    eo = list(itertools.chain(*eo))
    counter = 0
    order = []
    flag = True
    while 1:
        order.append(eo[counter])
        if counter == num-1:
            break
        if flag:
            counter += vcmax
            flag = False
        else:
            counter -= (vcmax-1)
            flag = True
    sorder = [tuple(order[x:x+2*vcmax]) for x in xrange(0,len(order),2*vcmax)]
    for s in xrange(0,kmax-1):
        sorder[s] = list(itertools.chain(*sorder[s]))
    vlb = pyglet.graphics.Batch()
    color = []
    for s in xrange(0,len(sorder)):
        color.append([])
        for v in xrange(2,len(sorder[s]),3): #start on 3 vertice, the altitude
            val = (sorder[s][v]/theight+1)/2.0 #map in range 0,1
            for colr in cs.hsv_to_rgb(0.5,1,val): #get rgb
                color[s].append(int(colr*256))
    for s in xrange(0,len(sorder)):
        sorder[s] = tuple(sorder[s][0:3] + sorder[s] + sorder[s][-3:])
        color[s] = tuple(color[s][0:3] + color[s] + color[s][-3:])
        vlb.add(2*vcmax+2, pyglet.gl.GL_TRIANGLE_STRIP, None, ('v3f', sorder[s]), ('c3B', color[s]))
    return vlb

world = trimesh(90,90,10, 0,-400, narray, 50)

def display():   
    glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
    glShadeModel(GL_SMOOTH)
    glLoadIdentity()

def reshape():
    glShadeModel( GL_FLAT )
    glMatrixMode(GL_PROJECTION)
    glLoadIdentity()
    gluPerspective(120., win.width / float(win.height), .1, 1000.)
    glMatrixMode(GL_MODELVIEW)
    glTranslatef(0.,0.,-100.)
    glRotatef(0,1,0,0)

a, b, = win.width/2, win.height/2 #this stores mouse position
k = 0.01 #sensitivity
gx = 0 #this stores the angle
gy = 0 #this also stores the angle
dx = 0 #default xpos #positive is rightward
dy = 0 #default ypos #positive is forward
dz = 0 #default zpost #positive is up

@win.event
def on_mouse_motion(x, y, dx,dy): #No effect?
    k = 7
    global a, b
    a += dx*k
    b += dy*k
    return pyglet.event.EVENT_HANDLED

fps_display = pyglet.clock.ClockDisplay()

@win.event
def on_draw():
    global a, b, k, gx, gy, dx, dy, dz
    display()
    reshape()
    win.clear()
    #
    glMatrixMode(GL_MODELVIEW)
    gx += -k*(a-win.width/2) #if mouse on right side of screen, gx will be positive. If left, gx is negative
    gy += -k*(b-win.height/2) #if mouse on top of sceen, gy will ne positive. If bottom, gy is negative
    a, b, = win.width/2, win.height/2 #reset positions so scene does not move
    #
    glRotatef(gy,1,0,0)
    glRotatef(-gx,0,0,1)
    glTranslatef(-dx, -dy, -dz)
    #
    world.draw()

def move(dt):
    global gx, gy, dx, dy, dz
    held = kb[pyglet.window.key.W]
    kt = 3
    if held:
        heading = (gx+90) % 360
        pitch = (270-gy) % 360
        dz += math.sin(pitch*math.pi/180)*kt
        comp = math.cos(pitch*math.pi/180)*kt
        dx += comp*math.cos(heading*math.pi/180)
        dy += comp*math.sin(heading*math.pi/180)
   
pyglet.clock.schedule_interval(move,1.0/120.0)

pyglet.app.run()

demo
smoothpy

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #332 on: January 03, 2014, 11:57:58 PM »
My first actual legitimate program release!

imgget v1.0

I've created a simple file downloader that you eternally run to download files for you! It is meant for files that update frequently (more frequently than once a day) so that it will catch them when you're not around.

Operation:

1. Configure the config.txt file so that it links to a url you want to download and after the comma, include the filename that you want it to download as before the timestamp. So if you put "foobar" then it will download as "foobar20140101_1830", so you may want to put "foobar_".
2. You will unfortunately manually have to download your first file (save it with the correct filename in which you want all the other files), and enter the timestamp into the list in lastdownload.txt. Not too difficult though.
3. Run, and it should do all the work for you. It shows simple status messages to let you know everything is (or is not) working okay.
« Last Edit: January 04, 2014, 12:03:49 AM by atomic7732 »

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #333 on: January 13, 2014, 05:57:10 PM »
I am amaze at my PHP skills.

I got a webpage to read an advisory from an entirely different website, and find a storm name and number, which i can now modify to look more presentable.

Code: [Select]
<?php
copy
('http://weather.noaa.gov/pub/data/raw/wt/wtio30.fmee..txt','wtio30.txt');

$handle fopen("wtio30.txt""r");
if (
$handle) {
    while ((
$line fgets($handle)) !== false) {
        echo 
$line;
        echo 
'<br />';
    }
} else {
    echo 
'Error reading file.';
}

echo 
'<br /><hr /><br />';

$handle fopen("wtio30.txt""r");
if (
$handle) {
    while ((
$line fgets($handle)) !== false) {
        if (
preg_match('/1.A/'$line$matches)){
            echo 
$line;
            echo 
'<br />';
            echo 
substr($line4).'<br />';
            
preg_match('/ [0-9] /'$line$sn);
            echo 
'Storm Number: '.$sn[0].'<br />';
            
preg_match('/\([A-Z]+\)/'$line$name);
            
$temp substr($name[0], 1strlen($name[0])-2);
            echo 
'Storm Name: '.substr($temp01).strtolower(substr($temp1));
        }
    }
} else {
    echo 
'Error reading file.';
}

?>

I'm going to use this to create a database which just records all updates to relevant cyclone files... basically pywx online. And no python.

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #334 on: January 13, 2014, 06:15:33 PM »
woah that color coding...

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #335 on: January 13, 2014, 08:00:25 PM »
apparently SMF has it built in to color PHP

blotz

  • Formerly 'bong'
  • *****
  • Posts: 813
  • op pls
Re: Coding
« Reply #336 on: January 14, 2014, 04:53:41 PM »
atomic tell me where do you even put php

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #337 on: January 14, 2014, 07:44:44 PM »
PHP works on a web server to create a webpage that isn't necessarily the same every time you call it. A normal HTML webpage will always be the same unless someone changes the way it's written so that it looks different, but 5 years down the road, someone can request the page and it will be the same as it was 5 years ago. PHP can do stuff like look up information in databases or other dynamic things that may have the page end up different based on user input or something that changed.

I don't know if you wanted an explanation on PHP as well, but there you have it.

blotz

  • Formerly 'bong'
  • *****
  • Posts: 813
  • op pls
Re: Coding
« Reply #338 on: January 15, 2014, 04:46:47 AM »
ok so it's a thing to put on a webpage k thx

tuto99

  • *****
  • Posts: 533
  • Baba Booey
Re: Coding
« Reply #339 on: January 15, 2014, 12:44:46 PM »
ok so it's a thing to put on a webpage k thx
You have learned a lot

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #340 on: January 18, 2014, 05:34:00 AM »
erosion simulator. requires matplotlib and numpy

you can use it without matplotlib or numpy if take them out in the code and delete the display and update functions. then you'd need someone to plot it for you

before

after


usage guide to erosim

first we make the landscape with random walk

Code: [Select]
import random
walker = 0
path = []
for x in xrange(0,400):
walker += random.random()-random.random()
path.append(walker)

now we have to match the end of the path up to the beginning. if we don't there is a large discontinuity and the erosion will exponentially grow bigger, causing an overflow error.

Code: [Select]
slope = (path[-1]-path[0])/400
we subtract the slope so that it becomes flat
Code: [Select]
for x in xrange(0,400):
t[x] -= slope*x
   
and now set the minimum to zero.

Code: [Select]
t = [val-min(t) for val in t]
all of the above can be written into a function of course, so you don't have to type it in every time.
navigate to the correct directory with os and execute the file
Code: [Select]
import os
os.chdir('c:\\users\\kolkon\\desktop\\erosim')
execfile('erosim.py')

then we create a world. we must give the elevation map, flow rate, erosion rate, evaporation rate, rock hardness, and rain amount in that order.
here is a sample that works relatively well. notice we use list(t).

Code: [Select]
g = World(list(t), 0.2, 10.0, 0.01, 5.0, 1.0)   
lets take a look
   
Code: [Select]
g.display()   
now we run the simulation, eroding and making it rain periodically. make sure there isn't already too much water when you add more or everything will flood
Code: [Select]
for x in xrange(0,10000):
if not(x%100) and max(g.water) < 1:
g.rain()
g.erode()
   
lets see how it went

Code: [Select]
g.update()
and there we are.

Bla

  • Global Moderator
  • *****
  • Posts: 1013
  • The stars died so you can live.
Re: Coding
« Reply #341 on: January 18, 2014, 06:32:04 AM »
Map War 3 - now with erosion!

blotz

  • Formerly 'bong'
  • *****
  • Posts: 813
  • op pls
Re: Coding
« Reply #342 on: January 18, 2014, 06:33:39 AM »
barbaria erosion >:D

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #343 on: January 18, 2014, 07:23:05 AM »
instead of subtracting the water from and right and water from the left, a typo caused only water to be subtracted from the right which caused a strange flow error. that and other things fixed here.

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #344 on: January 19, 2014, 08:29:46 AM »
draft of a new project. puts organisms in a environment and evolves their code

Code: [Select]
import numpy as np
import math

class Cell:
    def __init__(self, position, world):
        #self.center = position
        #self.blocks = [position]
        #world.cells.append(self)
        #self.energy = world.initenergy
        #self.inst = Code()
    def step(self):
        #try:
            #self.Code.run()()
        #except:
            #pass
    def grow(self):
        #while 1
            #pick random cell
            #check if it has any edges
            #pick random edge and fill
    def move(self):
        #call position on world class
    def make(self):
        #mutate code, place in front
    def eat(self):
        #of the objects touching, dissapear one
        #add energy
    def atk(self):
        #of the objects touching, dissapear one cell
        #make sure to call that cell and split
        #convert to food
    def see(self):
        #touching search
        #also farthest search
    def turn(self):
        #rotate direction so that movement is different

class Code:
    oplist = [Cell.grow, Cell.move, Cell.make, Cell.eat, Cell.atk, Cell.see]
    loops = ['for', 'if', 'while']
    conditional = [Cell.size, Cell.energy, Cell.sight, Cell.danger]
    other = ['break', 'pass']
    def __init__(self):
        self.topnode = {None:[]}
    def codeiterate(self, function):
        #queue root node
        #while queue
            #pick node in queue
            #visit node and apply function to it
            #add all children to queue
            #delete node from queue
    def shuffle(self):
        #use codeiterate and call random.shuffle on a list
    def prune(self):
        #probablility of deleting random code with codeiterate
    def mutate(self):
        #swap out and switch 'for', 'if', and 'while' while codeiterating
    def clone(self):
        #duplicate lines while codeiterating
    def mix(self, other):
        #not sure yet
    #def dfs(self, graph, string):
        #string = graph.keys()[0]
        #if not(graph[string]):
            #return nullstring
        #for k in graph[string]:
            #indented add dfs(graph, string)
        #return string
    def run(self):
        #inst = exec(dfs(self.topnode))
        #return inst

class World:
    def __init__(self, size, initenergy, decayrate, toroidal = True, solar):
        #self.grid = np.zeros(size)
        #self.cells = []
        #self.initenergy = initenergy
        #self.decayrate = decayrate
        #self.toroidal = toroidal
        #self.solar = solar
    def move(self, obj):
        #check collision
        #move if possible
    def step(self):
        #for each cell
        #run decay, solar, etc
        #run code

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #345 on: January 23, 2014, 10:47:17 PM »
Progress on my map plotter!

Unfortunately it took 10 minutes just to generate this pretty crude image:

note to self: latitude lines
« Last Edit: January 24, 2014, 07:37:26 AM by atomic7732 »

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #346 on: January 24, 2014, 05:30:46 AM »
no code?

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #347 on: January 24, 2014, 07:29:27 PM »
27 line alternative solution. readme in the code. generates full world resolution map in about 10 minutes if you have enough ram (that'll be 180,000 pixels by 90,000 pixels)

sample image here at scaling = 300 of europe
https://www.dropbox.com/s/8q471ypp753sv17/wmap.png

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #348 on: January 24, 2014, 09:39:08 PM »


It looks pretty horrible but I'm not sure how to improve it...

Darvince

  • *****
  • Posts: 1842
  • 差不多
Re: Coding
« Reply #349 on: January 24, 2014, 11:28:10 PM »
Is that super doge Haiyan

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #350 on: January 25, 2014, 12:56:12 AM »
suppose you had some unknown code, statement, or function that you wanted to run. the source of this might come from a variety of places. randomly generated code, user input code, mathematical expressions from an irc bot -- and on and on.

now also suppose you wanted to run a lot of these codes or functions. perhaps 10 or a hundred or a thousand at a time. this is useful if you had an irc calculator that needed to keep reading the chat while processing.

well, i've written a function to do so. the three arguments are time, the function, and the arguments:

Code: [Select]
import os
from multiprocessing import Process
import threading
import time

def timeoutprocess(_time, function, arg):
    if __name__ == "__main__":
        import time
        p = Process(target=function, args = arg)
        p.start()
        now = time.time()
        while 1:
            if time.time()-now > _time and p.is_alive():
                try:
                    p.terminate()
                except:
                    pass
            if not(p.is_alive()):
                break

def timeoutthread(time, function, arg):
    t = threading.Thread(target=timeoutprocess, args = (time, function, arg))
    t.start()


here's a short explanation of how this works.
we run the function on a separate process. process and not thread because
1. there is no way to terminate threads in python, but there is a way to terminate processes
2. takes advantages of more cores/parallelization
we can't simply spawn off a new process, we have to spawn the new process in a new thread. this is because there is a while loop inside the thread that keeps track of the time. when time is up, the process is killed. If we didn't use threads, we'd have to compute everything serially and not simulatenously.

and here's an example usage.
Code: [Select]
execfile('tof.py')

def wait(null):
    while 1:
        pass

def test():
    if __name__=="__main__":
        print 'hi', time.time()
        timeoutthread(1,wait,(1,))
        print 'bye', time.time()

for x in xrange(0,10):
    test()
we have a function test. inside that function, we do some things, then spawn off a separate process to take care of a task. note that you need if __name__=="__main__":
this is because python's structured so that the process can only access the code and know what to run by importing the file. however, if it imports the file, it'll also execute the line of code that spawns itself, and processes will keep spawning more processes until the system freezes up.


vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #351 on: January 31, 2014, 11:16:26 PM »
here's an update to my key derivation function. there was a problem before where i was simply adding everything up, so a person could save a lot of memory just by keeping the last sum, which is bad. (because if they save a lot of memory, that means they can run it concurrently on thousands of threads, which defeats the entire point).

however the algorithm is still compute bound. at ten thousand rounds, the algorithm takes about half a minute to run on a single thread and 10MB of memory. I'm shooting for a time of about a second and 1gb of memory

what i plan to do is to reduce the computation time by not hashing all of the previous values, but only some, depending on the value of the last hash

yes i'm aware algohash is the most lame, undescriptive, and incorrect name for a key derivation function ever

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #352 on: January 31, 2014, 11:42:19 PM »
i've been working on this.

https://github.com/osmotischen/evolve5


vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #353 on: January 31, 2014, 11:55:11 PM »
short piece of code that counts how much code you've written.

Code: [Select]
import os
sum1,sum2=0,0
for f in [val for val in os.listdir(os.curdir) if '.py' in val]:
    with open(f, 'r') as file:
        d = file.readlines()
        sum1 += len(d)
        sum2 += len([val for val in d if ('#' not in val and val)])
print sum1, sum2

the two numbers are total amount and total amount excluding whitespaces and comments

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #354 on: February 06, 2014, 04:36:45 PM »
this plots a histogram from a one dimensional dataset. except it's smoooooth

more like a density function

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #355 on: February 06, 2014, 08:06:12 PM »
after a few hours of experimentation and a new folder dedicated to plotting i've completed my minimalist density curve plotter in python.

i'll be adding an option for standard deviation and mean. right now zero looks pretty ugly

atomic7732

  • Global Moderator
  • *****
  • Posts: 3848
  • caught in the river turning blue
    • Paladin of Storms
Re: Coding
« Reply #356 on: February 06, 2014, 08:08:03 PM »
wow that looks cute

Bla

  • Global Moderator
  • *****
  • Posts: 1013
  • The stars died so you can live.
Re: Coding
« Reply #357 on: February 07, 2014, 06:25:48 AM »
MatLab has many of those functions, actually a very cool program and programming language. It has an std() function for finding standard deviation.
I once made a time dilation visualizer in it and a program for finding a minimum in functions of 1 and 2 variables.

But I'm pretty sure it isn't free, I got a license from university.

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #358 on: February 07, 2014, 12:49:12 PM »
ha. the module i (and most other people) use for plotting, matplotlib, is basically just matlab's plotting functions imported to python


you should upload that time dilation ponyer
« Last Edit: February 07, 2014, 12:53:26 PM by vh »

vh

  • formerly mudkipz
  • *****
  • Posts: 1140
  • "giving heat meaning"
Re: Coding
« Reply #359 on: February 07, 2014, 12:53:51 PM »
I ALWAYS THOUGHT DILATION WAS SPELLED DIALATION LIKE "TURN THE DIAL TO THE TIME MACHINE QUICK!"