Trade Number Crunching

So.  This is hard to explain.

For more than two weeks, I have been putting all my energy towards a rebuild of one key part of my trade table.  I'm not changing any of the math, I'm only adjusting the manner in which the excel file is built.  The part I'm fixing corresponds to the calculation of transport against references.  See the link for a file that can be downloaded; the key page is the "New Master" tab.

Now, why anyone would care, I don't know; but I've been working on this day and night to the exclusion of all else, as long as 12 hours a day when I have the whole day.  I started on Friday the 15th.  I'm still working on it.  I guess this has been 62-68 hours of continuous work.  Why?  Because it is manually rewriting more than 40,000 excel entries.

Previously, the "master" page that accounted for all production for all market cities was a simple chart, 892 products across the top of the page and 1,067 market cities down the left side.  This made a chart of 951,764 cells, mostly empty, with about 5% of the cells filled with references.  When I wanted to calculate the distance effect on references, as shown on the wiki link, I had to multiply the whole list of cities against all the cells (because it wasn't feasible to pick and choose cells is such a large field), which was getting incomprehensibly slow and file crashing for my computer to manage.

And, of course, the problem was only going to get worse, because I had to add more markets: Burma, all of Great Britain, Iceland and part of Africa - a total of another 150 markets.  Something had to change.

So, now I have built the table so that it sorts each kind of product into its own individual chart; the calculation is made automatically when the distances are input into the file (as can be seen on the downloaded link) and added together.  I will then add another page that correlates the individual products so that it can be compared with the prices table.

This actually reduces my mechanics by one whole file, as it compresses two files into one (don't worry about it, it would take me too long to explain those files).  Plus it adds information, as now the user can go down the list and see where the largest amount of imported value is coming from.

I offer this for geeks only. Most of you just won't care, won't get anything out of the excel file, won't understand why I'm doing this.  But it is definitely better.

I'm pretty close now to getting all the data recalculated; but most of that is on another file, not linked, sorted but not actually pasted onto the new folder.  It has been a long, long effort.  I'm pretty tired of just crunching numbers.  But this is the reason I haven't been writing much; unlike making monsters, which screams blog material, rewriting numbers into new fields and organizing data, not so much.

I'll be done this soon, however, and looking for something else to do.  I think, in the future, it will be easier to add new markets to my trade tables, and that is what I want right now.  Earlier, I had to pretty much do big chunks of new data all at once, basically waiting to collect a chunk before updating everything.  Now I should be able to do one city at a time, very easily.

I'm really glad about this.