Welcome, Guest

Author Topic: How to judge the accuracy of a simulation?  (Read 4336 times)

emarksmi

  • ****
  • Posts: 58
How to judge the accuracy of a simulation?
« on: December 21, 2016, 01:15:56 PM »
First, let's say that I'm having a blast with this program.  So much to experiment with.

Let's start with the fact that I'm an engineer and while I don't do a lot of programming anymore, I used to do some simulation work so I understand issues involved with adjusting time steps, sensitivity to initial conditions, accumulated error, etc.

So, I have a simulation that I started, save when I want to go play with something else, and keep reloading to pick up where it left off.  What I did was start with the default solar system simulation that is running when you open the program, took Jupiter's orbit and stuck it at a 30 degree inclination, and then cut the orbital velocity in half (this resulted in a perihelion just outside Venus' orbit and an aphelion at Jupiter's original orbit).  I then just let it run to see how the system evolved.  It has been fascinating to see what has happened.  After ~2400 years of simulated time (7d/s target time step which runs at about 6.5d/s), the asteroid belt has been completely disrupted, Mars is now on a 70 degree inclined orbit with an perihelion of about .5 AU and an Aphelion of about 4.5 AU, and Earth is on a 30 degree inclined orbit with a perihelion of about .15 AU and an Aphelion of about 2.8 AU (among lots of other changes).  Changes in Saturn's orbit are just starting to become apparent (inclination is SLOWLY increasing and the orbit is just starting to show some elongation).  I've had it running at home since I left for work this morning and can't wait to see what has happened while I've been gone.

I've looked at the Sim tab and see some figures regarding the current accuracy of the simulation, but am not sure hot to interpret them.  The two key numbers seem to be a velocity number that jumps around between 0.3 and 0.4 m/s and a position number that is measured in lots of km.  I'm not sure what they are telling me.  Are they total error?  Some computation of error in each time step?  They don't look like accumulated error because they aren't increasing.  Are they summed over every object in the simulation or are they reduced in some way?

I'm curious as to how to know when the simulation is threatening to blow up or has turned into complete garbage.

Thanks for all your help.

Physics_Hacker

  • *****
  • Posts: 441
Re: How to judge the accuracy of a simulation?
« Reply #1 on: December 21, 2016, 02:41:36 PM »
I don't know what the figures you're talking about mean, I don't even know which ones you're talking about, but the simulation should stay quite accurate for a very long time due to the timestep having a limiter that keeps the calculations from going too fast (and therefore less accurate) so since the calculations are more accurate it'll stay that way for a long time, you shouldn't have to worry about the total error causing the simulation to "blow up" or turn "into complete garbage" for a very long time if ever due to the timestep limiter (thats why the actual timestep is half a day a second slower than your target timestep)

Sorry if this helped none

emarksmi

  • ****
  • Posts: 58
Re: How to judge the accuracy of a simulation?
« Reply #2 on: December 21, 2016, 03:05:04 PM »
It didn't not help.  I'm familiar with programming variable time steps and that is an art in and of itself.  I know that the results to this point seem reasonable.  Nothing has done anything that is completely surprising once I think about it (although I did need to think about why Mars got torqued into the orbit it is currently in).

I am very happily surprised that a program that is billed as a "game" is so wonderfully accurate and stable.  I'm not really worried about the simulation crashing at this point, but I know that the indications for simulation health are there, I just don't know how to interpret them.

Physics_Hacker

  • *****
  • Posts: 441
Re: How to judge the accuracy of a simulation?
« Reply #3 on: December 21, 2016, 04:15:15 PM »
It didn't not help.  I'm familiar with programming variable time steps and that is an art in and of itself.  I know that the results to this point seem reasonable.  Nothing has done anything that is completely surprising once I think about it (although I did need to think about why Mars got torqued into the orbit it is currently in).

I am very happily surprised that a program that is billed as a "game" is so wonderfully accurate and stable.  I'm not really worried about the simulation crashing at this point, but I know that the indications for simulation health are there, I just don't know how to interpret them.

They shouldn't be too inaccurate, unless you turn the timestep ridiculously high, past what the limiter can, well, limit, but I guess that's kinda obvious...at least it's not as bad as US1, in that version, it doesn't take much to make Mercury fly out of the solar system.

emarksmi

  • ****
  • Posts: 58
Re: How to judge the accuracy of a simulation?
« Reply #4 on: December 21, 2016, 04:56:18 PM »
They shouldn't be too inaccurate, unless you turn the timestep ridiculously high

Yep, but that shouldn't be a huge issue if you just keep in mind what you are trying to simulate.  If you are looking for effects in the inner solar system, 1 y/s is WAY too big.  If you are looking at galaxy collisions, 1 y/s is WAY too small.  Just think about the scales and speeds and start with something appropriate.

Physics_Hacker

  • *****
  • Posts: 441
Re: How to judge the accuracy of a simulation?
« Reply #5 on: December 21, 2016, 10:43:57 PM »
They shouldn't be too inaccurate, unless you turn the timestep ridiculously high

Yep, but that shouldn't be a huge issue if you just keep in mind what you are trying to simulate.  If you are looking for effects in the inner solar system, 1 y/s is WAY too big.  If you are looking at galaxy collisions, 1 y/s is WAY too small.  Just think about the scales and speeds and start with something appropriate.

Yeah, exactly. following that makes sure the calculations stay accurate and probably could give pretty accurate results even after millions of years of simulation time (well, as long as you're willing to wait that long...) especially because the devs have designed US2 to where we can be impatient and still keep things semi-accurate, but that just means those that are patient are very very accurate

scruffygamer

  • ****
  • Posts: 51
Re: How to judge the accuracy of a simulation?
« Reply #6 on: January 25, 2017, 07:57:39 PM »
You can go into the simulation's settings and turn up or down the accuracy.

Turn it up for a bit more accuracy in the simulation.

Turn it down for silliness and see planets fling around as you crank up the speed.

emarksmi

  • ****
  • Posts: 58
Re: How to judge the accuracy of a simulation?
« Reply #7 on: January 25, 2017, 10:30:47 PM »
You can go into the simulation's settings and turn up or down the accuracy.

I understand that I can play with the accuracy, but there are running indications of what the current error in the simulation is.  I'm trying to find information on how to interpret those indications.