Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 .. 21 :: [one page] |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
5
|
Posted - 2011.09.06 18:16:00 -
[1] - Quote
EVE Marketeer is a website in the spirit of EVE Central and EVE Marketdata, just better than both (if I may say so myself).
It gathers data from both of the services above, and combines this with the data that is coming from our own uploader, guaranteeing the most up to data data you can get anywhere. We also do some special magic to try and make sure idiotic orders are filtered, but we're not always successful.
An API is provided for developers in three different formats: XML, CSV and JSON. Most of the data can be retrieved by way of the 6 simple calls we provide.
In addition to providing market information, EVE Marketeer is filled with some very handy tools to help you earn money, and keep track of your money, these include:
- Manufacturing cost calculator
- Mining profits
- Trade Finder
- Shopping List
- Best items to trade between regions
- Order Tracking (do you have the highest/lowest sell/buy order)
- Asset Search
- Transaction list / profits view
- Awesome forum signature images based on character data
This is only the beginning. We aim to provide you with all sorts of information. Please let me know if you need anything, and I'll see if I can implement it. There is a very useful feedback form on the website that keeps track of all requests that have been made, and how much they are loved/needed. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
5
|
Posted - 2011.09.08 17:46:00 -
[2] - Quote
With the fixes/additions to the API, I hope that the your orders information is being updated correctly again (though I imagine it might still go wrong if your orders last more than 7 days). I'll fix that this weekend.
Due to RL time constrains since working fulltime (I think you can guess around what time that happened), I'll probably only be able to work on EM during the weekends, though I'll of course respond to posts here all the time. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
5
|
Posted - 2011.09.10 16:32:00 -
[3] - Quote
Today, I made some modifications to how the item information page is loaded. It now loads all tabs dynamically to save on overal page loading time.
Overall I'm currently working on some optimization, since I've noticed the website become increasingly slower with an increasing amount of data.
In addition, I might be looking for people whom are interested in helping develop EVE Marketeer. I'm not certain yet, but since I noticed my own availability for actual work dropping, I am open to any interest people might have.
EDIT: Also hard at work on converting the website to the new API key system :) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.09.10 18:59:00 -
[4] - Quote
Just started getting these errors within the last hour:
Quote:Debug: Retrieved new files Debug: process: skipping unknown key_type GetRegionBest Debug: 10000002 Debug: upload: uploading data_type=GetOrders; upload_type=orders; Debug: upload: uploading to http://eve-marketdata.com/api/upload.phpDebug: upload: uploading to http://www.evemarketeer.com/api/uploadDebug: upload: uploader.send = 200, OK Debug: upload: response= Warning: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /var/www/core/lib/adodb5/drivers/adodb-mysql.inc.php on line 362 1 Error: Upload unknown problem: Warning: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /var/www/core/lib/adodb5/drivers/adodb-mysql.inc.php on line 362 1 Debug: returned from upload function Debug: removing the cachefile Debug: Processed all files, clearing queue EDIT: BTW, they are anomalous. It seems to fail for awhile, then work for awhile, then it starts to fail again. Connection issues with the database server? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
5
|
Posted - 2011.09.10 19:04:00 -
[5] - Quote
It was a known problem. I had some issued upgrading MySQL. For some reason it determined that it shouldn't start anymore, and for just as obscure a reason, it suddenly works again. It should be stable now (if a bit slow).
UPDATE: Okey, **** me, it is NOT stable. I'll be working on fixing that. Until I say it's fixed, please don't worry if you have intermittent (or constant ) connection issues. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.09.10 19:11:00 -
[6] - Quote
Callean Drevus wrote:In addition, I might be looking for people whom are interested in helping develop EVE Marketeer. I'm not certain yet, but since I noticed my own availability for actual work dropping, I am open to any interest people might have. I'm interested in helping out in whatever way that I am able. I am familiar with all of the tech that you have linked at the bottom of the site, except that I have used neither PHEAL nor YAPEAL before. (I write my own libraries.)
Examples of my past work include various tools for EVE University. The ones still in active use:
- Hangars Automated Inventory System (HAIs), *internal use only* - Agent Rangefinder - http://util.eveuniversity.org/Missions/AgentRangefinder/ - Planetary Interaction Taskforce Webtools - http://util.eveuniversity.org/PiBear/ - simple test mapping engine, not currently used in any production sites - http://util.eveuniversity.org/JSMapEngine/ |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
5
|
Posted - 2011.09.11 07:42:00 -
[7] - Quote
I would be mostly looking at people whom can develop stuff. In the end, I'm aiming to make EVE Marketeer the best, but that will probably happen faster with some more people to share the load with.
Are you familiar with mercurial repositories? And do you have any ideas of what you would/could built into EVE Marketeer, or what it should be in the long term?
Different note, it has been in EMK for some time already, but I"ve never really come around to building an interface element for it. It is a view to allow you to look at the price levels of PI components and determine whether it's really worth it to develop it to that level 4 item. http://www.evemarketeer.com/item/piprice/Enriched_Uranium
You'll have to enter the item names manually (substitute underscores for spaces). I'll eventually add this in a tab under the item screen.
Oh, and the database should be functional again now. It took a rather long while and some rebuilding, but everything is as before :) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.09.15 03:43:00 -
[8] - Quote
I think the sql server might have just died :P
Oh and I'd be happy to help with development where/when I can. Haven't got the foggiest idea what a mercurial repositories is though :P |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.09.15 17:37:00 -
[9] - Quote
Mercurial is a version control system. I've never used it myself (I use Git), but wouldn't imagine it'd be difficult to adapt to its commands if you've used any other kind of version control (RCS, CVS, Subversion, Git, etc.)
EDIT: Oh yes, and confirming, the site still seems to be down. :) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
7
|
Posted - 2011.09.15 19:00:00 -
[10] - Quote
Ok :) I'm using mercurial for the EM versioning so you'd have to use it. Not that my versioning is exceptionally decent (in fact, it's horrible, but at least it's versioning), but it allows for easy updating of the website (and using it for that is bad practice too, but it's a hobby project, so it's allowed). I'll see whether I can create a general account for EM developers on my hosted repository, otherwise I might migrate it to the server itself.
In any case, it must've been something I was doing to the server, it never let me down like that before... it does seem to have killed all the tables though. That'll be repairing myisam tables for the rest of the night. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
7
|
Posted - 2011.09.17 23:17:00 -
[11] - Quote
This needs to be on top!
And I've got a good excuse, I added the utilization of customizable keys. Now you guys all need to enter your information again, but it's worth it. Everything is done automatically. Mainly thanks to Dragonaire and Yapeal though :) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
7
|
Posted - 2011.09.20 21:17:00 -
[12] - Quote
Ah, guys/gals, I think I've come up with a really nice feature now (or well, it existed elsewhere of course, but now I bring it to the masses).
It's called 'My Blueprints', and it simply shows you a list of all blueprints you have, with their material levels (as based on the industry jobs you've started/completed) and based on the prices in your current region, whether it is worth it to build them. Nice red and green row backgrounds show you what is worthwhile at a glance.
I'm going to be using this one often :)
Also, the order uploads have finally caught up to reality, so all market prices should be current again now. I also made the checking of false or idiotic orders MUCH more robust, taking into account such things as the most common prices, excluding outrageous orders, while retaining a margin of flexibility which should allow for prices to change (and be played with quite a bit). In the future this may be refined to not simply throw the odd orders away, but flag them and hide them until someone wants to see them (and even search for them specifically, who knows what nice stuff might be there, it's also excluding honest mistakes now ).
Oh, and last but not least, the site now remembers what region you had selected last time you were there, so no more automatically switching back to Jita! [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Muscaat
EVE Markets
1
|
Posted - 2011.09.20 23:30:00 -
[13] - Quote
As of about 15 minutes ago uploads were failing:
Quote:2011-09-21 00:19:48.016 WARN: [EVEMarketeerUploader] EVE Marketeer upload failed with error Fatal error: Call to a member function metadata() on a non-object in /var/www/mod/global/global.php on line 20
(First line of the log is mine)
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
8
|
Posted - 2011.09.21 05:48:00 -
[14] - Quote
Me knows, it is horrible. Fix under way. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
8
|
Posted - 2011.09.21 16:08:00 -
[15] - Quote
Should be fixed now. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Scrapyard Bob
EVE University Ivy League
0
|
Posted - 2011.09.22 02:17:00 -
[16] - Quote
The API is throwing errors all day again.
LOGSQL Insert Failed: insert into adodb_logsql (created,sql0,sql1,params,tracer,timer) values( NOW(),?,?,?,?,?) No database selected
Warning: Cannot modify header information - headers already sent by (output started at /home/admin/domains/coronum.nl/public_html/dev/evemarketeer.com/core/lib/adodb5/adodb.inc.php:482) in /home/admin/domains/coronum.nl/public_html/dev/evemarketeer.com/mod/api.php on line 151 |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
9
|
Posted - 2011.09.22 06:46:00 -
[17] - Quote
Where exactly do you get these errors? The whole site runs on the same database, and the site seems to be running fine.
Update: My guess is you're using an outdated version of the uploader or an API reference to the old site. Try using www.evemarketeer.com instead of evemarketeer.com. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Scrapyard Bob
EVE University Ivy League
0
|
Posted - 2011.09.22 15:30:00 -
[18] - Quote
That was indeed it, my ImportXML() was still pulling from "evemarketeer.com" instead of "www.evemarketeer.com". |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
11
|
Posted - 2011.09.24 13:08:00 -
[19] - Quote
BIG update today!
I added pretty much everything that you could do to yourself for corporations. That means you can now view assets, orders, transactions and blueprints (profitability) for your corporation.
This information is ATM only accessible when you are the corporation director.
Of course, to make it all work, you need to have added a corporation key to your account. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Emma Royd
Maddled Gommerils
0
|
Posted - 2011.09.25 06:53:00 -
[20] - Quote
All I get when I go to http://www.evemarketeer.com/ is a blank page atm. |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
11
|
Posted - 2011.09.25 08:12:00 -
[21] - Quote
I'm on it!
UPDATE: Fixed it has been [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingyoku
Griffin Capsuleers Ad-Astra
0
|
Posted - 2011.09.25 15:44:00 -
[22] - Quote
Well this is kind of embarrassing, I usually use keepass for my passwords and it seems to have not saved my password when I setup my account. I also noticed there's no password reset on the site, any plans to implement that? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
11
|
Posted - 2011.09.25 22:12:00 -
[23] - Quote
Shingyoku wrote:Well this is kind of embarrassing, I usually use keepass for my passwords and it seems to have not saved my password when I setup my account. I also noticed there's no password reset on the site, any plans to implement that?
Ah, yes, this is originally an improvement on a corporation website I made, and if anyone lost their password there I could simply reset it myself. I'll send you a new password soon, and make a way to easily reset it yourself (which is going to be interesting, as I do not save any email adresses by default). [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.09.26 01:16:00 -
[24] - Quote
Just send them an ingame eve-mail. No emails required :P |
Shingyoku
Griffin Capsuleers Ad-Astra
0
|
Posted - 2011.09.26 04:24:00 -
[25] - Quote
Domi Etymology wrote:Just send them an ingame eve-mail. No emails required :P
I really like this idea. Use the trust feature in the browser or something. |
Gretzaki
The New Federation Detrimental Imperative
0
|
Posted - 2011.09.26 14:21:00 -
[26] - Quote
Site is down again, it's hard to use the data that is being collected when you can't access it :( |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
12
|
Posted - 2011.09.26 16:52:00 -
[27] - Quote
Gretzaki wrote:Site is down again, it's hard to use the data that is being collected when you can't access it :(
I am very sorry about this. I assure you, I am not doing it on purpose and am trying to track down what is going wrong. Previously, I knew it had something to do with an unlimited number of threads spawning, but now it just seems to decided to die every 1.5 days.
I can't always be online however, so it may take some time to restart if it isn't working (actually, the remote reboot doesn't work either. It just totally loses connection. Only a complete re-initialization of the VM brings it back online).
And... The logs show nothing! :S sorry, but it may take a while, or it may be fixed tomorrow, that's all I can really say about it [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingyoku
Griffin Capsuleers Ad-Astra
0
|
Posted - 2011.09.26 19:59:00 -
[28] - Quote
Callean Drevus wrote:Gretzaki wrote:Site is down again, it's hard to use the data that is being collected when you can't access it :( I am very sorry about this. I assure you, I am not doing it on purpose and am trying to track down what is going wrong. Previously, I knew it had something to do with an unlimited number of threads spawning, but now it just seems to decided to die every 1.5 days. I can't always be online however, so it may take some time to restart if it isn't working (actually, the remote reboot doesn't work either. It just totally loses connection. Only a complete re-initialization of the VM brings it back online). And... The logs show nothing! :S sorry, but it may take a while, or it may be fixed tomorrow, that's all I can really say about it
Welcome to website admin, isn't it lovely? :D |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
13
|
Posted - 2011.09.27 16:52:00 -
[29] - Quote
Shingyoku wrote:Domi Etymology wrote:Just send them an ingame eve-mail. No emails required :P I really like this idea. Use the trust feature in the browser or something.
I would love to, but sending Ingame mail cannot be automated, so if you are okey with waiting for me to send a mail, thats fine :P [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingyoku
Griffin Capsuleers Ad-Astra
0
|
Posted - 2011.09.27 17:29:00 -
[30] - Quote
Callean Drevus wrote:Shingyoku wrote:Domi Etymology wrote:Just send them an ingame eve-mail. No emails required :P I really like this idea. Use the trust feature in the browser or something. I would love to, but sending Ingame mail cannot be automated, so if you are okey with waiting for me to send a mail, thats fine :P
Oh I'm ok with that for now, just saying for the future reset system. |
|
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.09.28 02:14:00 -
[31] - Quote
I strongly recommend against using IGB trust to disseminate lost passwords. It is one of the easier things to spoof. Registering an email address on account creation would be the ideal "easy" solution.
EDIT: However, sending in-game EVEmails for password reminder/reset could be secure if the account has a CAK registered with character info access. But as stated above, it cannot be automated. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
13
|
Posted - 2011.09.28 09:46:00 -
[32] - Quote
Turhan Bey wrote:I strongly recommend against using IGB trust to disseminate lost passwords. It is one of the easier things to spoof. Registering an email address on account creation would be the ideal "easy" solution.
EDIT: However, sending in-game EVEmails for password reminder/reset could be secure if the account has a CAK registered with character info access. But as stated above, it cannot be automated. I am not planning on using the IGB for any authentication whatsoever :)
It might just be safer and best to do those few times it happens manually. Until CCP builds in automatic sending of IG mails (spam ftw?). [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Scrapyard Bob
EVE University Ivy League
0
|
Posted - 2011.09.29 01:51:00 -
[33] - Quote
Quote:LOGSQL Insert Failed: insert into adodb_logsql (created,sql0,sql1,params,tracer,timer) values( NOW(),?,?,?,?,?) Table './admin_icras/adodb_logsql' is marked as crashed and last (automatic?) repair failed
http://www.evemarketeer.com/statistics/uploads/24
This is showing up at the top of a few pages as well as in the upload tool. |
Dragonaire
Corax. PURgE Alliance
11
|
Posted - 2011.09.29 03:47:00 -
[34] - Quote
Looks like the DB isn't keeping up and crashing for some reason I'd make sure it's not filling up it's log space. I've ran into default configs where every few days that was happening to me as well. There's some settings in my.cnf you should check and make sure it's not configured to keep them forever only like last 24 hours or so. Having thre extra query logging on also add a lot of extra load so except if you just turned it on to figure out what's going wrong I'd have it off on a production server. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.09.29 17:54:00 -
[35] - Quote
I've noticed that when receiving these errors, it will sometimes work without fault, but then it will generate errors continuously for an even longer time. The ratio varies greatly, but seems to average perhaps 10 errors to 1 success.
I've also seen the Upload Backlog with greatly skewed values against EVE-MarketData. For example, as I write this, I see the backlog as:
EVE Marketeer: 2 EVE Marketdata: 127 EVE Central: 0
Just a guess, but perhaps it is the update rate from the EMD feed that is throttling the database? Can it be slowed down to see if doing so has an effect on the error rate? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
13
|
Posted - 2011.09.29 18:15:00 -
[36] - Quote
Hey, thank you for your replies and help trying to get it fixed.
I made a post a few hours ago, but it seems it didn't get trough. First of all, I've installed logging software, that at least gives me an idea of what happens. If you feel interested, you can view the server stats via Munin
As you can see, there isn't anything shocking there, nothing suddenly happens, except a sudden spike in the usage of one disk. I've yet to find out what is causing this.
To clear two things up. It is not the error you've been seeing about logging, at least it seems very unlikely. That is just due to me neglecting to recover that table after the last crash (it's pretty much always writing to the log table, so every crash kills it :P).
It does not seem to be the database either, as I've been monitoring that, and it steadily keeps at about 5-10 connections doing queries a second, which isn't enough to kill it.
[img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Gretzaki
The New Federation Detrimental Imperative
0
|
Posted - 2011.09.30 03:35:00 -
[37] - Quote
Callean Drevus wrote:Hey, thank you for your replies and help trying to get it fixed.
Update: It might be google, trying to index the website :P
Update II: The logs seem to be fine. No huge files there.
If you really think it might be a google index, try using a robots txt to ignore that directory of the site.
Also I would be interested in helping with development in any way I can. Please let me know.
and btw the backlogs are piling up....
EVE Marketeer 6355 EVE Marketdata 25 EVE Central 0 |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.09.30 06:53:00 -
[38] - Quote
my uploader keeps giving me this:
Error: Upload unknown problem: Warning: mysql_connect(): Lost connection to MySQL server at 'reading initial communication packet', system error: 111 in /var/www/core/lib/adodb5/drivers/adodb-mysql.inc.php on line 362 1
? |
Vessper
Indicium Technologies Hephaestus Forge Alliance
0
|
Posted - 2011.09.30 14:40:00 -
[39] - Quote
Region XML download is giving errors again:
Warning: mysql_connect(): Lost connection to MySQL server at 'reading initial communication packet', system error: 111 in /var/www/core/lib/adodb5/drivers/adodb-mysql.inc.php on line 362
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
14
|
Posted - 2011.09.30 20:13:00 -
[40] - Quote
Ok, I basically removed every server package that had anything to do with serving up pages and reinstalled the latest versions, and sometimes even different packages (I'm now using nginx instead of apache, and running php as cgi instead of module). In any case, I hope it solves some of the problems.
Also, I might need to start processing EMD orders in the same way I now process EMK orders, since that seems to be the fastest way, and I now get an immense backlog of EMD orders :P or I could just work on making order processing faster.
Gretzaki, expect an ingame mail about helping development soon. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
|
Scrapyard Bob
EVE University Ivy League
0
|
Posted - 2011.10.01 02:29:00 -
[41] - Quote
Callean Drevus wrote: Also, I might need to start processing EMD orders in the same way I now process EMK orders, since that seems to be the fastest way, and I now get an immense backlog of EMD orders :P or I could just work on making order processing faster.
As in, making the upload tool send them directly to EMD instead of passing them through the EMK site? That would be good, at least EMD would get its updates even if EMK is having trouble (or vice-versa).
(Tonight, in order to update market data, I'm using the EMD market-scanner page, because the EMK page button to start scanning is not showing up.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
14
|
Posted - 2011.10.01 07:30:00 -
[42] - Quote
It actually does directly send them to EMD, even before it sends them to EMK. So in that regard everything plods along pretty well even if the website is down (though the tool will keep complaining about not being able to upload to EMK, it should have uploaded to EMD fine) :P
I'll take a look at the scanner page, maybe something to do with the move. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Scrapyard Bob
EVE University Ivy League
0
|
Posted - 2011.10.04 15:17:00 -
[43] - Quote
The scanner page is definitely still having issues, if it does come up, it eventually ends up with a gateway timeout error - which means my noble attempt to leave an alt logged in overnight comes to naught.
On the feature request side:
http://www.evemarketeer.com/statistics/uploads - It would be better if this page was sorted by region name in alphabetical order, instead of by RegionID
http://www.evemarketeer.com/statistics/regions - same request, please sort by region name
http://www.evemarketeer.com/item/itemprice/Oxygen_Isotopes - On the regional prices tab, it would be useful if there was an indicator as to how current the data is for each region. Maybe presented as "The Forge (3 hrs)" or "Domain (2 days)" - The orders tab, doesn't let me look at other regions |
Scrapyard Bob
EVE University Ivy League
0
|
Posted - 2011.10.04 15:24:00 -
[44] - Quote
One other question - does the EMK uploader send data to EVE Central as well?
(The fact that you send data to EMD in addition to EMK is a key reason I use your uploader.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
14
|
Posted - 2011.10.04 18:32:00 -
[45] - Quote
Scrapyard Bob wrote:The scanner page is definitely still having issues, if it does come up, it eventually ends up with a gateway timeout error - which means my noble attempt to leave an alt logged in overnight comes to naught. On the feature request side: http://www.evemarketeer.com/statistics/uploads- It would be better if this page was sorted by region name in alphabetical order, instead of by RegionID http://www.evemarketeer.com/statistics/regions- same request, please sort by region name http://www.evemarketeer.com/item/itemprice/Oxygen_Isotopes- On the regional prices tab, it would be useful if there was an indicator as to how current the data is for each region. Maybe presented as "The Forge (3 hrs)" or "Domain (2 days)" - The orders tab, doesn't let me look at other regions
Thank you for your feedback.
Sorting on the regions page makes sense, I'll fix that.
Sorting on the uploads page is already by name, only from left to right and up to down, instead of up to down left to right.
The itemprice page currently has a field that shows you how recent the data was. I agree it might be useful to add it to the regional tab. I do wonder what you mean by, the orders tab does not let me look at other regions though. I think your issue is mainly with having to reload the entire page to switch regions, while it could be done faster? Or that you would like to quickly look at a different region without changing your default?
In regards to the uploader, it does not upload to EVE Central. Nor do I have plans to support it in the future. I do understand Muscaat of EVE Markets made an uploader that uploaded to all of our websites (EC, EMK and EMD), though the one that I can download from his website at this moment does not work (at least on my PC).
Lastly, the uploader page, I have no idea why it would be giving gateway timeout errors. I'll test it myself and see what I can dig up. It it not really acceptable that people leave their computers on all night to upload, only to find that it crashes halfway trough. Otherwise I'll just make the refresh AJAX too, and have it automagically refresh it's list if it detects no activity. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
17
|
Posted - 2011.10.13 22:06:00 -
[46] - Quote
Did some minor fixes, and implemented caching today (for home and for statistics pages). Caching might be a bit erratic at the moment. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.10.19 00:11:00 -
[47] - Quote
getting an error when trying to log in at the moment, and this error message
Server Time: 2011-10-19 00:10:43 Warning: Invalid argument supplied for foreach() in /var/www/core/lib/dwoo/plugins/personal/region_list.php on line 28 |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.10.19 10:08:00 -
[48] - Quote
Get well soon! |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.19 17:18:00 -
[49] - Quote
When you're feeling better.
1) http://www.evemarketeer.com/player/account Warning: implode(): Invalid arguments passed in /var/www/mod/player.php on line 75
2) You also might want to turn the order history page when looking at an item into a paged list, or shorten it. Right now it takes a long time to load and is very long.
3) Shorten/page the upload history page. Also some of mine are showing -1 for the credits earned. ex: Assault Missile Launcher II, Assault Missile Launcher I Blueprint, Ship Scanner II
4) On the mission statistics page, it says I need to add a character to the account, when I have and it has pulled the API for it. Maybe say what permissions from the API are needed when showing that error. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
22
|
Posted - 2011.10.19 18:35:00 -
[50] - Quote
Thank you for your kindness I'm still feeling unwell though.
Which is one of the reasons I'm uploading the entire EVE Marketeer codebase to Github. Previously, I already added a few people to the mercurial repository I was using before, but this seems immensely easier. Besides, everyone will be able to contribute instead of just a few.
I'll also expose access to the live database to a few people that I trust not to give away private information.
This will hopefully allow updates to happen even if I'm not feeling ok .
The only thing I'm struggling with is providing a copy of the database. I could allow a complete download, but at the moment it's about 45 GB, which is probably a tad much for most people. Anyone have any ideas? [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.19 20:28:00 -
[51] - Quote
Callean Drevus wrote:Thank you for your kindness I'm still feeling unwell though. Which is one of the reasons I'm uploading the entire EVE Marketeer codebase to Github. Previously, I already added a few people to the mercurial repository I was using before, but this seems immensely easier. Besides, everyone will be able to contribute instead of just a few. I'll also expose access to the live database to a few people that I trust not to give away private information. This will hopefully allow updates to happen even if I'm not feeling ok . The only thing I'm struggling with is providing a copy of the database. I could allow a complete download, but at the moment it's about 45 GB, which is probably a tad much for most people. Anyone have any ideas?
Maybe just include active orders, or just ones that have been seen in the past x days in the download. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
22
|
Posted - 2011.10.19 21:24:00 -
[52] - Quote
Hmm, git is being recalcitrant as usual. I wonder why I still try... Maybe just stay with mercurial... [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.19 21:30:00 -
[53] - Quote
Callean Drevus wrote:Hmm, git is being recalcitrant as usual. I wonder why I still try... Maybe just stay with mercurial...
I don't remember what it was but with another project that I think was on google code, people could upload patches that the devs would then incorporate into the main code if they thought it was good. I don't remember what system it was with, but maybe that would be helpful allowing anyone to submit code patches to you.
It seems I about doubled/tripled the length of the feedback page on the site now =] |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
22
|
Posted - 2011.10.19 21:51:00 -
[54] - Quote
Woot! EVE Marketeer was on github. And then I noticed I had some production passwords in the source that should have been hidden. Oh well, I'll give up for today. Tomorrow will probably see me simply committing the damn thing to bitbucket. I do (again) not feel like dealing with git ever again.
And yes, both of those websites allow people to commit patches which I can then merge into the main codebase. That's kind of the point
You won't be able to edit the code on the site itself though, but using mercurial should be relatively painless (as opposed to git).
EDIT: Regarding my sig, yeah, but I seem to not be able to edit it anymore. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.19 21:58:00 -
[55] - Quote
Callean Drevus wrote:Woot! EVE Marketeer was on github. And then I noticed I had some production passwords in the source that should have been hidden. Oh well, I'll give up for today. Tomorrow will probably see me simply committing the damn thing to bitbucket. I do (again) not feel like dealing with git ever again. And yes, both of those websites allow people to commit patches which I can then merge into the main codebase. That's kind of the point You won't be able to edit the code on the site itself though, but using mercurial should be relatively painless (as opposed to git). EDIT: Regarding my sig, yeah, but I seem to not be able to edit it anymore.
I haven't used github much. I'm used to SVN. I'm assuming you will post a link to the repo as soon as you are done setting it up?
Edit: just curious. What are the machine specs this is run off of? I know it's ubuntu and running nginx. Apache backend or just nginx? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
23
|
Posted - 2011.10.20 06:41:00 -
[56] - Quote
Try appending ?debug=3 to any URL if you want to see loadtime and queries.
The specs are: i3 Intel processor 4 Gb RAM 2 1 Gb SATA disks in raid 1 100 Mbit uplink
Those are probably the most relevant ;) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.20 16:22:00 -
[57] - Quote
Callean Drevus wrote:Try appending ?debug=3 to any URL if you want to see loadtime and queries.
The specs are: i3 Intel processor 4 Gb RAM 2 1 Gb SATA disks in raid 1 100 Mbit uplink
Those are probably the most relevant ;)
Ok. I was thinking of something more along the lines of it showing all the time in a way that doesn't mess up the page layout. ex. http://chubaka.dyndns.biz/~ci/ipb_stats.png |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
24
|
Posted - 2011.10.20 16:29:00 -
[58] - Quote
I know you were :) this is just the best it can do at the moment. Havent found a decent way to incorporate the stats into the page yet. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.20 17:00:00 -
[59] - Quote
Callean Drevus wrote:I know you were :) this is just the best it can do at the moment. Havent found a decent way to incorporate the stats into the page yet. Ok. I thought the screenshot would help give you an idea of what I meant and an idea for a way to incorporate it into the site.
How about this? http://chubaka.dyndns.biz/~ci/evemarketeer_stats.png (I just grabbed the code from the IPB Forums site and stuck it in there using your theme.) http://chubaka.dyndns.biz/~ci/stats_code.txt (if you like it, there is the code/links to images. I think they are the famfamfam silk icon ones though) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
24
|
Posted - 2011.10.20 17:28:00 -
[60] - Quote
Lol, nice going there
Don't you think it's a lot of space (distraction) for just the stats though? That has been what has kept me from implementing it this way. I would do it if I had something else to show in that bar, but right now, it's just a waste of space (though I doubt many people would mind, spending 10% of the (visible) page height on stats feels wrong to me)
That said, think up something useful to go on the left side of that box, and it would be ok ;) maybe we should also show number of users active or something [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
|
Scrapyard Bob
EVE University Ivy League
37
|
Posted - 2011.10.20 18:05:00 -
[61] - Quote
I see no issue with it being in the bottom bar, or in tiny text at the bottom of the screen (I'd prefer it down there, out of the way rather then in a left/right column which would shrink screen width). |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
24
|
Posted - 2011.10.20 18:08:00 -
[62] - Quote
Scrapyard Bob wrote:I see no issue with it being in the bottom bar, or in tiny text at the bottom of the screen (I'd prefer it down there, out of the way rather then in a left/right column which would shrink screen width).
Ah, yes, I hoped it was clear I was referring to the left side of the bar, not left side of the page ;)
I'd been thinking about tiny text at the bottom myself, but there does not seem to be a color that will work for both the light and dark parts of the background ;) I might just implement something along Controller's design. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.20 18:17:00 -
[63] - Quote
Callean Drevus wrote:Lol, nice going there Don't you think it's a lot of space (distraction) for just the stats though? That has been what has kept me from implementing it this way. I would do it if I had something else to show in that bar, but right now, it's just a waste of space (though I doubt many people would mind, spending 10% of the (visible) page height on stats feels wrong to me) That said, think up something useful to go on the left side of that box, and it would be ok ;) maybe we should also show number of users active or something Maybe put it on the same line as the powered by images, or make it optional and add a option to the user settings. I'm sure you could make it smaller than what I did, I just didn't feel like messing around with css more than I had to. On a side note, is it just me or does the site logo text look a bit choppy?
If I had more time I would consider helping you out some with this, unfortunately I don't. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
24
|
Posted - 2011.10.20 19:04:00 -
[64] - Quote
Ok, so I finally got a decent version of the EVE Marketeer source uploaded. As I suspected, using Mercurial was immensely easier than Git (I'm not bashing Git, I'm just saying I'm not cut out for it, version control should just stay out of your way).
https://bitbucket.org/aeolun/eve-marketeer/overview
The database can also be downloaded (only the structure for now, populate it with data yourself :P) from here.
Now go ahead and let me know about those glaring SQL injection attacks, cross site scripting and the fact that the comment form secretly allows you to enter whatever HTML you want [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.20 20:42:00 -
[65] - Quote
Callean Drevus wrote:Ok, so I finally got a decent version of the EVE Marketeer source uploaded. As I suspected, using Mercurial was immensely easier than Git (I'm not bashing Git, I'm just saying I'm not cut out for it, version control should just stay out of your way). https://bitbucket.org/aeolun/eve-marketeer/overviewIt is released under the WTFPLThe database can also be downloaded (only the structure for now, populate it with data yourself :P) from here. Now go ahead and let me know about those glaring SQL injection attacks, cross site scripting and the fact that the comment form secretly allows you to enter whatever HTML you want UPDATE: The live version of EVE Marketeer is now the bitbucket version, making for easy updates. Alright. I edited the code for the bottom. How is this? http://www.evemarketeer.cmu.ois-eve.info/ |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
25
|
Posted - 2011.10.20 20:52:00 -
[66] - Quote
I had to get used to it for a bit, but I think it looks pretty nice. You did a clone of the code? If so, can you submit it? Will have to figure out how bitbucket handles that one way or another. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.20 20:59:00 -
[67] - Quote
Callean Drevus wrote:I had to get used to it for a bit, but I think it looks pretty nice. You did a clone of the code? If so, can you submit it? Will have to figure out how bitbucket handles that one way or another.
I actually just downloaded it as a zip. http://www.evemarketeer.cmu.ois-eve.info/changes-10-20-2011.zip those are the things I changed there. There might have been a better way to do that, but I did that without actually figuring out what everything in the code does.
Edit: If you got that zip file when it contained main.php, download it again. It was supposed to have init.php instead of main.php |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
27
|
Posted - 2011.10.22 14:53:00 -
[68] - Quote
Ah lol, I didn't see your extra messages because they were edits to the previous post.
The text doesn't actually look better for me. Quite the contrary, it's got a fuzzy haze over it now (antialiasing gone wrong).
Ah, the difference between mercurial and SVN (I believe you used that previously) is that after you commit it to your local repository, you also have to push your changes to the server. I do not believe you can do this at the moment because you do not have write permissions on the repository.
To push it, I believe you have to follow steps 2, 4 and 5 on this page: http://confluence.atlassian.com/display/BITBUCKET/Sharing+Code+in+Bitbucket
Of course, I'll probably be implementing the zipped up file before that happens :P [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.10.22 16:14:00 -
[69] - Quote
Callean Drevus wrote:Ah lol, I didn't see your extra messages because they were edits to the previous post. The text doesn't actually look better for me. Quite the contrary, it's got a fuzzy haze over it now (antialiasing gone wrong). Ah, the difference between mercurial and SVN (I believe you used that previously) is that after you commit it to your local repository, you also have to push your changes to the server. I do not believe you can do this at the moment because you do not have write permissions on the repository. To push it, I believe you have to follow steps 2, 4 and 5 on this page: http://confluence.atlassian.com/display/BITBUCKET/Sharing+Code+in+BitbucketOf course, I'll probably be implementing the zipped up file before that happens :P
Ok. I don't see the issue with the new text. Maybe it is something different with the browsers. The thing I was trying to take care of was the jagged edges on the slanted parts, such as the V, M, R... For me the only difference with the new one is cleaner edges and a slight size difference. |
Scrapyard Bob
EVE University Ivy League
62
|
Posted - 2011.10.23 06:32:00 -
[70] - Quote
Once the site comes back up I have a few questions...
- On the manufacturing information, what assumptions are you making with regards to T1 manufacturing of items? Are you assuming a reasonable ME level and Production Efficiency V?
- Have you considered trying to estimate T2 invention costs (such as what EVETools does)?
|
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
28
|
Posted - 2011.10.23 08:56:00 -
[71] - Quote
By default, I'm using the production efficiency level of the character selected at that moment. If none is selected, a PE of 0 is assumed. The PE level is at 0 by default too, but you can chance that one with the inputs above the information itself.
I have not considered it for a second, as son as I saw it I knew it should be in EVE Marketeer, but the recent troubles have given me other things to do. There are just way too many things that should still be added ;) be assured it will eventually make it's way into EMK though :)
Besides that, I also picked up on your wish to have a multi-region/market hub price, which I agree would make a good estimate of value, so I'll be adding that too once the problems are gone. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
29
|
Posted - 2011.10.30 19:28:00 -
[72] - Quote
EVE Marketeer is back up!
For those of you whom did not notice yet
We're currently working on getting the backlog of order uploads processed as fast as possible, and making good progress. Some issues were detected with the new process control system, but I'm pretty sure I'll be able to fix those as time goes along.
Now on to adding some new features!
In addition, my apologies for the sadly extended downtime. Hopefully the new system will scale better. If not, I'll have to look at increasing the server capacity. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Uskos
New Ceres Economics and Logsitcs
1
|
Posted - 2011.10.31 21:18:00 -
[73] - Quote
I very much support this product! BTW it seems like i cannot download the uploader. |
Quinn Fellthorn
EVE University Ivy League
1
|
Posted - 2011.11.01 14:26:00 -
[74] - Quote
The site is up but it doesn't seem to be updating. API queries still return data up to two weeks old, and the statistics page shows a sizeable backlog with no uploads in the last 24 hours. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
31
|
Posted - 2011.11.01 16:58:00 -
[75] - Quote
That is because the backlog isnt finished updating yet (going from oldest till newest). The uploads in the last hour are actually sort of a misnomer in that it shows the amount of uploads processed that contain data in the last hour.
That said, there IS a problem where the backlog is not being updated, but a very minor one that will be fixed this evening :) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Ameron Phinard
0
|
Posted - 2011.11.02 16:15:00 -
[76] - Quote
Uskos wrote:I very much support this product! BTW it seems like i cannot download the uploader.
yes -- it does look like all the links to the Uploader are broken. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
31
|
Posted - 2011.11.02 18:05:00 -
[77] - Quote
Seen it, confirmed it, and fixed it now. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
31
|
Posted - 2011.11.03 21:02:00 -
[78] - Quote
In addiiton, the website should now stay up. My idea of using PHP for threads was useful, but PHP doesn't do garbage collection, so the server kept running out of memory after half a day. I'll be monitoring the situation now. I hope it stays steady. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Zaepho
Phoenix Propulsion Labs RED Citizens
2
|
Posted - 2011.11.03 21:18:00 -
[79] - Quote
Callean Drevus wrote:In addiiton, the website should now stay up. My idea of using PHP for threads was useful, but PHP doesn't do garbage collection, so the server kept running out of memory after half a day. I'll be monitoring the situation now. I hope it stays steady. I've seen PHP Threading used quite well. The trick seems to be, start the main thread, dole out work to child threads, keeping X threads busy at a time until the work is done. In your case you may want to kill and restart the main thread every so often as well if you're seeing memory leakage issues. |
Shirah Yuri
Allied Assault Universal Constant Alliance
7
|
Posted - 2011.11.04 07:37:00 -
[80] - Quote
Callean Drevus wrote:In addiiton, the website should now stay up. My idea of using PHP for threads was useful, but PHP doesn't do garbage collection, so the server kept running out of memory after half a day. I'll be monitoring the situation now. I hope it stays steady.
http://php.net/manual/en/features.gc.php |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
31
|
Posted - 2011.11.04 17:47:00 -
[81] - Quote
Ok, true, PHP does garbage collection, but as far as documentation goes, that comment by the user was it... I do not mind using beta features, but even I have a limit on how far it can go.
Secondary reason was I got sick of having to restart all threads by hand at every update, so now they just restart themselves ;) the script dies, the memory is cleared, and the new script takes over. The only problem being the server still magically killing my processes... [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
31
|
Posted - 2011.11.06 11:13:00 -
[82] - Quote
Please all look forward to uploading to EVE Central with the EVE Marketeer uploader soon :) I expect it to be done this afternoon. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.06 21:17:00 -
[83] - Quote
Interesting, the top uploaders page shows my old number of uploads and points, yet my user summary page shows the new, reset, and much lower count. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
32
|
Posted - 2011.11.06 21:24:00 -
[84] - Quote
Actually, it wasn't reset. The user summary page is just deprecated and uses the wrong table to calculate the statistics.
UPDATE: Which is also the reason why it was removed in this update. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.06 21:42:00 -
[85] - Quote
Callean Drevus wrote:Actually, it wasn't reset. The user summary page is just deprecated and uses the wrong table to calculate the statistics.
UPDATE: Which is also the reason why it was removed in this update. Ok, well you might want to give people a place to get their upload key then.
Edit: maybe just fix it? It was sorta nice to be able to see that info. And it had the upload key on it. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
32
|
Posted - 2011.11.06 21:53:00 -
[86] - Quote
I will bring the information back, just not on that page. Will probably be integrated into either the UI, the Account page or another page.
The upload key can be retrieved from the upload page as well, which should be a pretty logical location. (UPDATE: though it might be a bit more prominent, it's very overshadowed by that large block of text underneath it) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.06 21:55:00 -
[87] - Quote
Callean Drevus wrote:I will bring the information back, just not on that page. Will probably be integrated into either the UI, the Account page or another page.
The upload key can be retrieved from the upload page as well, which should be a pretty logical location. (UPDATE: though it might be a bit more prominent, it's very overshadowed by that large block of text underneath it) Alright, you might want to change the text on the front page that says where to find it then.
Also, You know the "My Orders" page doesn't show any orders right? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
32
|
Posted - 2011.11.06 22:17:00 -
[88] - Quote
I cannot replicate the problem with the my orders page. Are you using a character without orders?
I moved the information that was available at the account page to... the account page :) you should be able to view it there now. And I updated the description on the homepage.
EDIT: Well, you'll have to wait for the uploader update. I'm not going to put it on the website without testing it with EVE Central as that was the whole purpose of the update. Sorry :P [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.06 22:52:00 -
[89] - Quote
Yes, my API had transactions and orders. It shows them on the API page on the marketeer site. I have used this API with these settings since I signed up, so it has had time. The orders page just doesn't show any orders, the transaction page says I need to log in or add a character. This is the API access mask: 29364610 |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
32
|
Posted - 2011.11.07 20:01:00 -
[90] - Quote
Controller Vrelk wrote:Yes, my API had transactions and orders. It shows them on the API page on the marketeer site. I have used this API with these settings since I signed up, so it has had time. The orders page just doesn't show any orders, the transaction page says I need to log in or add a character. This is the API access mask: 29364610
You were correct. Though I was too. It seems you had only enabled a select few items (which I agree is the point of customizable keys), but the website clearly stated that it could exhibit strange behaviour if a full key was not supplied.
And strangely behaving it does
Problem is, you need to add at least the CharacterInfo parameter, or the website will not even find your characters. This is obviously quite troublesome for the whole process of seeing whose orders you are currently looking at, and that is why the orders page didn't show anything.
I've now added a clearer warning to the website that will notify you if you haven't selected CharacterInfo, this should show up for you as well. Add that one to your key, and you'll probably be able to use EVE Marketeer within half an hour or so (as soon as the character list updates). [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.07 20:46:00 -
[91] - Quote
I just checked the API key, it already had character info enabled. The site shows the new error message saying it dosn't have it enabled though.
This is the api config page for what I see. http://ois-eve.info/files/eve_api.png |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
32
|
Posted - 2011.11.07 20:54:00 -
[92] - Quote
Damn, that's odd. I'll at it again. This screenshot helps! [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
32
|
Posted - 2011.11.07 21:09:00 -
[93] - Quote
Ok, it probably has to do with CharacterSheet being disabled then (damn CCP for naming them so close together).
It's now also mentioned on the key page. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.07 23:07:00 -
[94] - Quote
Ok, key is updated. EM shows the change. Orders is still empty, and assets and transactions still say I need to log in or add a character. |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.08 03:29:00 -
[95] - Quote
Found an issue, you seem to have reverse the order of sell orders. High prices are now at the top. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
33
|
Posted - 2011.11.08 07:41:00 -
[96] - Quote
Ok, just to let you know, when it works a character selection dropdown will show up on the left side of the secondary menubar. As long as it doesn't something is wrong, though I honestly couldn't say what is causing this problem now. I'll get to the bottom of this yet! [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.08 15:32:00 -
[97] - Quote
Callean Drevus wrote:Ok, just to let you know, when it works a character selection dropdown will show up on the left side of the secondary menubar. As long as it doesn't something is wrong, though I honestly couldn't say what is causing this problem now. I'll get to the bottom of this yet!
I think I know where you are saying there should be a dropdown box for the character, but there isn't one. I am waiting on my api to update to all characters. How often does the cron run to update API info so I know if it worked or not? |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.11.09 06:25:00 -
[98] - Quote
Traceback (most recent call last): File "uploader.py", line 1156, in "<"module">" File "uploader.py", line 84, in __init__ File "uploader.py", line 434, in check_version ValueError: too many values to unpack
had to put "" around the tags in module
but that's the error I'm getting in uploader.exe.log
and the uploader refuses to run now :( |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.11.09 08:42:00 -
[99] - Quote
Actually, don't worry, After rebooting, worked like a charm. Guess the new uploader just didn't like being run straight after installation. |
Strobist
Hello Kitty Space Marines Ev0ke
0
|
Posted - 2011.11.09 09:30:00 -
[100] - Quote
Uploader won't start. Log:
Quote:Traceback (most recent call last): File "uploader.py", line 1156, in <"modul"e> File "uploader.py", line 88, in __init__ File "uploader.py", line 118, in init_text AttributeError: 'Uploader' object has no attribute 'log' Traceback (most recent call last): File "uploader.py", line 1156, in <"module"> File "uploader.py", line 88, in __init__ File "uploader.py", line 118, in init_text AttributeError: 'Uploader' object has no attribute 'log' Traceback (most recent call last): File "uploader.py", line 1156, in <"module"> File "uploader.py", line 88, in __init__ File "uploader.py", line 118, in init_text AttributeError: 'Uploader' object has no attribute 'log' Traceback (most recent call last): File "uploader.py", line 1156, in <"module"> File "uploader.py", line 88, in __init__ File "uploader.py", line 118, in init_text AttributeError: 'Uploader' object has no attribute 'log'
After installing, I restarted.
Using Windows 7 x64 all Updates installed. |
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.11.11 00:27:00 -
[101] - Quote
try setting the uploader.exe to run as administrator mode. I have nfi why it needs it, but it seemed to clear up most of the headaches for me. |
Shingihada Rubik
University of Caille Gallente Federation
0
|
Posted - 2011.11.11 02:09:00 -
[102] - Quote
Doesn't work, I'm also getting the following error:
Traceback (most recent call last): File "uploader.py", line 1156, in <"module"> File "uploader.py", line 88, in __init__ File "uploader.py", line 118, in init_text AttributeError: 'Uploader' object has no attribute 'log'
|
Abdiel Kavash
Paladin Order Fidelas Constans
124
|
Posted - 2011.11.11 15:49:00 -
[103] - Quote
I am getting the above message too. I downgraded to 2.5.4EM, which seems to upload fine (the uploads are showing in both the uploader and on the website under My Market > Upload History. However, the website and the API still show an old value (as in, 3 days old). Is this a bug, or is there some delay in processing uploads? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
33
|
Posted - 2011.11.11 19:17:00 -
[104] - Quote
Umm, turning on EVE Central proccessing 1 day ago while I couldn't keep an eye on it turned out to be the wrong decision, so the queue is now more or less filled with lots of real and bogus updates from eve central. Problem is I don't know which are real so I have to let the entire queue finish. Or purge it which would remove all real updates as well. Will soon implement something to delete only queued items from a certain source.
I appreciate the uploader bug reports btw, I'll see if i can fix the problem this weekend (I'm a bit distracted by Skyrim, but this should be possible :) [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
33
|
Posted - 2011.11.12 19:07:00 -
[105] - Quote
Abdiel Kavash wrote:I am getting the above message too. I downgraded to 2.5.4EM, which seems to upload fine (the uploads are showing in both the uploader and on the website under My Market > Upload History. However, the website and the API still show an old value (as in, 3 days old). Is this a bug, or is there some delay in processing uploads?
The bug should be fixed in uploader version 2.6.1 which I've just released.
Furthermore, the site should be processing orders as normal again too though it might still take some time for it to catch up the backlog. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingihada Rubik
University of Caille Gallente Federation
0
|
Posted - 2011.11.13 15:40:00 -
[106] - Quote
Huh, I thought I posted something here earlier today, but I guess not.
Anyways, the new update fixed everything for me at least. So it's working flawlessly for me right now, however, when I booted up my computer this morning (Windows 7) I got a new error message, but just restarting the app worked. I'm just posting to let you know since the app still works fine.
Also, I constantly get the error about not being allowed to post HTML, so I just pasted the error to Pastebin: http://pastebin.com/S76D7eCU |
Scrapyard Bob
EVE University Ivy League
351
|
Posted - 2011.11.14 21:06:00 -
[107] - Quote
On the market scanner page - after a while (an hour or two), it stops giving commands to the in-game market window. Not exactly sure why, (maybe you're going up against some sort of throttle and you need to back off to 3.2 or 3.5 seconds).
The page is still running, it's just not looking up items on the market. |
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
39
|
Posted - 2011.11.14 22:30:00 -
[108] - Quote
Site throws an error when trying to see the prices on individual items. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
33
|
Posted - 2011.11.14 22:44:00 -
[109] - Quote
Shingihada Rubik wrote:Huh, I thought I posted something here earlier today, but I guess not. Anyways, the new update fixed everything for me at least. So it's working flawlessly for me right now, however, when I booted up my computer this morning (Windows 7) I got a new error message, but just restarting the app worked. I'm just posting to let you know since the app still works fine. Also, I constantly get the error about not being allowed to post HTML, so I just pasted the error to Pastebin: http://pastebin.com/S76D7eCU
Ah, yes, I've seen this too, but I cannot understand why it fails intermittently, it shouldn't be different on each startup. In addition, I only see it when running from the IDE, so I've yet to find out what's causing it.
@Scrapyard Bob: Does it stop looking up items (say, the market window does not constantly pop up anymore), or does it stop sending data? Since after a few hours, the entire market would be scanned, and EVE doesn't generate new cachefiles if everything is cached already (even if the uploader deleted the files).
If the second is true, was it always doing this? Or is it a new problem?
@Endeavour: I do not get any errors, where exactly are the problems occuring?
I've also been notified that the server has bad harddisks, which means they have to be replaced. In any case, the server would be moved soon anyway (probably this weekend), but you shouldn't notice anything this time (I've been careful about setting my TTL this time), and the true time I move will mostly will depend on how soon I finish or get bored of Skyrim [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingihada Rubik
University of Caille Gallente Federation
0
|
Posted - 2011.11.15 00:19:00 -
[110] - Quote
I've been seeing that hanging too, it appears it's just the market that isn't refreshed, the webpage keeps ticking in the background but I haven't checked with the uploader while playing if it still uploads.
Also, and this is mostly because I'm stupid: Is the delete cache option on for default? Because I've been mad at EVE Mentat the last three hours for not reading any cache files when I repeatedly tried to refresh it... and then it turns out EVE Marketeer Uploader removes the cache files after uploading. Hahaha. My rage was transferred to my desk via my head when I realized that. Ugh. |
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.11.15 04:10:00 -
[111] - Quote
saw something new from the uploader today (2.6.1)
Error: Upload Error: Unable to upload, data is too old, you have to upload within 15 minutes of the check. 2011-11-15 11:38:22: Uploaded 18054 in The Forge to EC 2011-11-15 11:38:22: Uploaded 18054 in The Forge to EMK Error: Upload Error: Unable to upload, data is too old, you have to upload within 15 minutes of the check. 2011-11-15 11:38:25: Uploaded 17367 in The Forge to EC 2011-11-15 11:38:33: Uploaded 17367 in The Forge to EMK Error: Upload Error: Unable to upload, data is too old, you have to upload within 15 minutes of the check. 2011-11-15 11:38:39: Uploaded 17235 in The Forge to EC 2011-11-15 11:38:40: Uploaded 17235 in The Forge to EMK 2011-11-15 11:38:42: Uploaded 19696 in The Forge to EMD 2011-11-15 11:38:42: Uploaded 19696 in The Forge to EC 2011-11-15 11:38:43: Uploaded 19696 in The Forge to EMK Error: Upload Error: Unable to upload, data is too old, you have to upload within 15 minutes of the check.
Has there been some delay built into the uploader now it's handling 3 sites. I'm not entirely sure how it's falling behind by 15 minutes, but......
btw, could those errors be a bit more specific, ie: which item ID, and to which site? I got nfi which uploads were failing from that description. |
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
39
|
Posted - 2011.11.15 04:16:00 -
[112] - Quote
Just went to another random item
http://www.evemarketeer.com/item/itemprice/3540
I get this
Quote:Fatal error: Uncaught exception 'Dwoo_Exception' with message 'Dwoo->get/Dwoo->output's first argument must be a Dwoo_ITemplate (i.e. Dwoo_Template_File) or a valid path to a template file' in /var/www/core/lib/dwoo/Dwoo.php:314 Stack trace: #0 /var/www/core/mod/template.php(33): Dwoo->get('core/tpl/item/i...', Array) #1 /var/www/core/mod/page.php(33): template_parse('item/itemprice....', Object(Request)) #2 /var/www/core/main.php(69): page->renderBody(Object(Request)) #3 /var/www/index.php(8): include('/var/www/core/m...') #4 {main} thrown in /var/www/core/lib/dwoo/Dwoo.php on line 314 |
Scrapyard Bob
EVE University Ivy League
351
|
Posted - 2011.11.15 13:46:00 -
[113] - Quote
Callean Drevus wrote: @Scrapyard Bob: Does it stop looking up items (say, the market window does not constantly pop up anymore), or does it stop sending data? Since after a few hours, the entire market would be scanned, and EVE doesn't generate new cachefiles if everything is cached already (even if the uploader deleted the files).
If the second is true, was it always doing this? Or is it a new problem?
The market window stops popping up.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
34
|
Posted - 2011.11.15 17:42:00 -
[114] - Quote
Lots of things to reply to :P
Yes, the delete cache option is deleted by default, because a big cache makes EVE slow and the uploader as well (since it has to check every file in the dir). If you let one of the programs delete the files eventually you should be fine though.
@Domi: That is evemarketeer telling you it doesn't want the data anymore I believe, but it might be the uploader. This might be happening because one of the sites is down, which would cause uploads to hang before a timeout occurs. Will look into fixing that (negating services that are down, or lowering timeout).
Endeavour, how do you go to the item? The URL has changed. The search function should work fin and I thought the browser should also work.
@Scrapyard: That is bad. I know it happened to me sometimes, and the only way to fix it is to reatart the IGB. Maybe it has to do with timeout. I might make timeout selectable, so you can put it more than 3 seconds. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingihada Rubik
University of Caille Gallente Federation
0
|
Posted - 2011.11.15 21:52:00 -
[115] - Quote
Callean Drevus wrote:Lots of things to reply to :P
Yes, the delete cache option is deleted by default, because a big cache makes EVE slow and the uploader as well (since it has to check every file in the dir). If you let one of the programs delete the files eventually you should be fine though. . Yeah, but it seems that is automatically ticks it off every time I reboot my computer (basically every morning). And this is really annoying because I will forget about it and then be angry when EVE Mentat doesn't find any cache files. Is there any way to turn this of by default, because EVE Mentat deletes the cache files anyways when it's done scanning.
Just convenience, but it's enough to make me a wee bit angry, haha. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
34
|
Posted - 2011.11.15 23:34:00 -
[116] - Quote
Well, it should remember your settings :P at startup the uploader tells you where your setting file is. If it isn't writing it correctly, maybe you can change it manually. If the problem persists, I'll try and see what I can do about it. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Shingihada Rubik
University of Caille Gallente Federation
0
|
Posted - 2011.11.15 23:51:00 -
[117] - Quote
Might have something to do with it crashing every time I reboot maybe? I just checked the config.ini file and it has correctly made delete cache = false. But yeah, I don't really know either. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
34
|
Posted - 2011.11.15 23:56:00 -
[118] - Quote
It could be writing an empty config file on crash, which would explain why the next startup reverts to default. I'll have to check that. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.11.16 07:57:00 -
[119] - Quote
which folder is the config.ini file in? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
34
|
Posted - 2011.11.16 14:27:00 -
[120] - Quote
C:\Users\*username*\AppData\Roaming\eve_marketdata\config.ini
Now that I think of it, the EVE Marketdata uploader might also be overwriting your perfect config if you are running it at the same time (or intermittently), I had that after I first started building the uploader.
I could of course just make a unique config file, but at first this was useful to make the switch easier for people. I'll consider changing the location of the configuration file at the same time as adding variable URL uploading to it. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
|
Scrapyard Bob
EVE University Ivy League
370
|
Posted - 2011.11.19 00:16:00 -
[121] - Quote
If I make an API query call with a wrong typeID like:
http://www.evemarketeer.com/api/info/37_40_36_0_11399_38_35_34_39/xml/10000030/sell_lowest5 or http://www.evemarketeer.com/api/info/37_40_36__11399_38_35_34_39/xml/10000030/sell_lowest5
I should still get back 9 rows. Instead, I only get back 8 rows in the above example. Ideally, that 4th row should simply have a zero in the sell_lowest5 and the datetime should be a date from a year ago (or the start of the epoch?).
|
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.11.19 17:04:00 -
[122] - Quote
Jita Prices API call is returning some very outdated information. (I've only tried that call, not sure if other API calls are functioning well or not.) Example for Mechanical Parts:
http://www.evemarketeer.com/api/jita_info/3689/xml
Select output, seems to reflect the state of the market during the datetime in question, but the datetime is rather old:
type_id: 3689 region_id: 10000002 station_id: 60003760 datetime: 2011-10-20 18:09:30
sell_lowest: 11239.98 sell_highest: 34000 sell_lowest5: 11243.358326225 buy_highest: 8501.04 buy_avg: 7665.2050780545 buy_lowest: 8501.04 buy_highest5: 8501.01826388197 |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
35
|
Posted - 2011.11.20 09:54:00 -
[123] - Quote
Huh? I replied to this yesterday...
In any case, will be fixed (the outdated information). And will be taken into consideration (the API queries), ideally you just wouldn't call the API with invalid information [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Scrapyard Bob
EVE University Ivy League
374
|
Posted - 2011.11.20 13:28:00 -
[124] - Quote
Callean Drevus wrote:In any case, will be fixed (the outdated information). And will be taken into consideration (the API queries), ideally you just wouldn't call the API with invalid information
I live to break things. In this particular case, I was setting up a sheet to track T1 ship construction prices and estimated margins. Since I don't have typeIDs for the new BCs yet, I had to just either leave those cells blank or put zeros in.
Zeros as a "invalid" return value are good, because you can easily exclude them with a CountIf() in most spreadsheets. And they don't screw up any Sum() functions. So if I pull 3 regions, I just do the calculation for the average price column as:
AvgPrice = Sum(A4:C4) / CountIf(A4:C4,">0")
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
35
|
Posted - 2011.11.20 14:16:00 -
[125] - Quote
Ok, well, since you ARE our best uploader, I think I might just make it accept invalid numbers as zero rows. Will see what I can do [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
35
|
Posted - 2011.11.22 20:48:00 -
[126] - Quote
It is monstrous what sort of an effect a specific server can have on your application. EVE marketeer is now running on hardware 4 times worse than the previous setup, but it's processing way faster than ever before... This could be awesome :D
First impressions of Storm on Demand are very good. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Scrapyard Bob
EVE University Ivy League
384
|
Posted - 2011.11.23 05:56:00 -
[127] - Quote
Tried using the new uploader again tonight, went back to EMD's. Hopefully you can pull the data from EMD
The issue that I have with the EMK uploader is that it's not very responsive to UI input when it's waiting for communication to the various websites to complete. I'm not sure what the fix there is, other then to split it into multiple threads, each thread uploading to a different site so that a single site being slow about processing doesn't impact the upload rates of the other sites. |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.11.23 21:13:00 -
[128] - Quote
Error return for current report bug, just in case it is helpful to you. :)
Call was: http://www.evemarketeer.com/api/jita_info/34/xml
Error returned: Fatal error: Uncaught exception 'Dwoo_Exception' with message 'Dwoo->get/Dwoo->output's first argument must be a Dwoo_ITemplate (i.e. Dwoo_Template_File) or a valid path to a template file' in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo.php:314 Stack trace: #0 /var/www/www.evemarketeer.com/core/mod/template.php(33): Dwoo->get('core/tpl/api/ji...', Array) #1 /var/www/www.evemarketeer.com/core/mod/page.php(33): template_parse('api/jita_info.t...', Object(Request)) #2 /var/www/www.evemarketeer.com/core/main.php(69): page->renderBody(Object(Request)) #3 /var/www/www.evemarketeer.com/index.php(8): include('/var/www/www.ev...') #4 {main} thrown in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo.php on line 314 |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.11.23 22:15:00 -
[129] - Quote
Scrapyard Bob wrote:Tried using the new uploader again tonight, went back to EMD's. Hopefully you can pull the data from EMD The issue that I have with the EMK uploader is that it's not very responsive to UI input
I've been noticing this myself. It didn't actually click that it the headaches I've been having with the uploader were occuring while it was waiting for communication to the different websites. Over the last couple of days, I've turned off uploading to eve-central, which helped a ton. Also, whatever you did with your server upgrade increased responsiveness a ton :)
Currently, if I leave 3 clients uploading data simultaneously for 15 minutes, I'll come back and the uploader window has just frozen. If I stop the eve clients from checking data, the client *might* eventually unfreeze and the log will scroll like crazy until it's caught up. Scrapyard Bob's suggestion sounds spot on, split the upload functions into different threads, so that the delay in waiting for responses from uploads isn't sending the whole uploader program to **** :) |
Scrapyard Bob
EVE University Ivy League
400
|
Posted - 2011.11.26 06:29:00 -
[130] - Quote
Got one tonight while looking up an item:
http://www.evemarketeer.com/item/itemprice/Memory_Augmentation_-_Improved
Fatal error: Uncaught exception 'Dwoo_Exception' with message 'Dwoo->get/Dwoo->output's first argument must be a Dwoo_ITemplate (i.e. Dwoo_Template_File) or a valid path to a template file' in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo.php:314 Stack trace: #0 /var/www/www.evemarketeer.com/core/mod/template.php(33): Dwoo->get('core/tpl/item/i...', Array) #1 /var/www/www.evemarketeer.com/core/mod/page.php(33): template_parse('item/itemprice....', Object(Request)) #2 /var/www/www.evemarketeer.com/core/main.php(69): page->renderBody(Object(Request)) #3 /var/www/www.evemarketeer.com/index.php(8): include('/var/www/www.ev...') #4 {main} thrown in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo.php on line 314 |
|
Scrapyard Bob
EVE University Ivy League
400
|
Posted - 2011.11.26 06:30:00 -
[131] - Quote
Domi Etymology wrote: I've been noticing this myself. It didn't actually click that it the headaches I've been having with the uploader were occuring while it was waiting for communication to the different websites. Over the last couple of days, I've turned off uploading to eve-central, which helped a ton. Also, whatever you did with your server upgrade increased responsiveness a ton :)
Currently, if I leave 3 clients uploading data simultaneously for 15 minutes, I'll come back and the uploader window has just frozen. If I stop the eve clients from checking data, the client *might* eventually unfreeze and the log will scroll like crazy until it's caught up. Scrapyard Bob's suggestion sounds spot on, split the upload functions into different threads, so that the delay in waiting for responses from uploads isn't sending the whole uploader program to **** :)
I'm now running both upload tools at the same time - and letting them fight over the cache files. If EMK's uploader gets side-tracked and starts timing out, the EMD upload tool picks up the slack. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
35
|
Posted - 2011.11.27 00:27:00 -
[132] - Quote
@Scrapyard: You are correct. The uploader is indeed very unresponsive while uploading. I'd been thinking about that problem myself, but have obviously not found a fix for it yet. Having the uploads handled in different threads would be nice (and very effective, at least in terms of UI responsiveness), but I so far have been unable to implement it.
@Turhan Bey: Thank you. Besided the old data issue, which I'm trying to fix in the station_info API (and then link jita_info to that), it now seems to be giving a template error (which is weird, as those functions should not look at templates anyway).
[img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
35
|
Posted - 2011.11.27 11:34:00 -
[133] - Quote
Ok, I think I fixed station information now (and thereby the jita_info API too). I also fixed a bug in the 'my orders' page which caused it to not be able to scan your orders. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.30 01:13:00 -
[134] - Quote
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 80 bytes) in /var/www/www.evemarketeer.com/core/mod/content/yapeal.php on line 107
While trying to load the history for Tritanium
Also
The "Total Bounties" and "Total Bounty Taxes" always shows the total amount regardless of what you set the time to. If you set it to 1 day, it should show totals for the past day. I don't know about the mission totals as I haven't run any in the time the site was able to grab logs for. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
36
|
Posted - 2011.11.30 07:52:00 -
[135] - Quote
Lol, ok, pagination here I come. It seems the total history for one item finally exceeds the limits of php.
And yes, I'll look into the mission page, it seems it should react to the time limit, but I'm not sure about it. Today is the first time in a while I finally have some time to actually do something again, so I hope to do some cool stuff today. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
0
|
Posted - 2011.11.30 22:43:00 -
[136] - Quote
1) Maybe limit the history tab to 90 orders per page, or 30 and go by months rather than a year.
2) After waiting about 30 seconds for the orders tab to load:
Warning: file_put_contents(var//classpath.cache.d15.php): failed to open stream: Permission denied in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Loader.php on line 86 Fatal error: Uncaught exception 'Dwoo_Exception' with message 'Could not write into var//classpath.cache.d15.php, either because the folder is not there (create it) or because of the chmod configuration (please ensure this directory is writable by php), alternatively you can change the directory used with $dwoo->setCompileDir() or provide a custom loader object with $dwoo->setLoader()' in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Loader.php:87 Stack trace: #0 /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Loader.php(104): Dwoo_Loader->rebuildClassPathCache('/var/www/www.ev...', 'var//classpath....') #1 /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Compiler.php(2703): Dwoo_Loader->loadPlugin('stationname', true) #2 /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Compiler.php(2494): Dwoo_Compiler->getPluginType('stationname') #3 /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Compiler.php(1981): Dwoo_Compiler->replaceModifiers(Array, 'var', 12) #4 /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Compiler.php(1 in /var/www/www.evemarketeer.com/core/lib/dwoo/Dwoo/Loader.php on line 87
I don't know if this is because you are still working on it, or something like the history page was doing yesterday (loading a long time then giving an error).
3) Make sure you mark the things as done on the feedback page, you missed pagination, clicking on search result to submit, etc.
4) The checkbox for the signature won't save, it always stays empty and the sig stays private.
4) I like the new tables at the top of the items page and the other layout/design/css changes.
5) The uploader is screwed up. It stopped uploading at 301, then stopped, no errors, just stopped. Restarting it didn't fix it, nor did disabling all but EMK.
6) The uploads aren't being processed? |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
1
|
Posted - 2011.12.01 03:40:00 -
[137] - Quote
Alright, I see you made it show the totals for the time set, sorta. I say sorta because I noticed a counting problem.
http://compu.redshift-gaming.net/images/js/EMK_mission.png Just noticed the offset in the numbering, oh well, you get the point. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
37
|
Posted - 2011.12.01 21:28:00 -
[138] - Quote
Most of those things fixed. And put the server in temporary reinforced mode while I mess with the database. [img]http://www.evemarketeer.com/player/sig/511049420[/img] Want images? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.04 16:37:00 -
[139] - Quote
Ok, so today I built a few of the features that have been on the wishlist for a while.
In addition, I also updated the database to use the new Crucible database.
The new things are: - Show a aggregate of the stats for the top 5 regions. This means the website takes an average of the values (for example lowest sale price) in the 5 regions with the most orders. This should help counter any market manipulation attempts a bit. An API for this feature is coming soon. - Allow filtering by station in the orders list, as well as clean the list up a bit by hiding most of the orders you might not want to see. - Add an addition row to item information that shows the changes in price the past day, and also shows an estimate of the buy/sell ratio that is used in some computation on the website - Importing from EVE Central and EVE Marketdata have been enabled again and do not seem to be giving any problems, except that those two websites are not sending a lot of data (not any data atm, which I cannot understand).
You might've noticed I've been working hard on fixing stuff that was on the website for a while. This is because I felt that the overal quality of some things on the website was a bit low. New features have been moved to the background a bit, but are certainly not forgotten. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.04 17:52:00 -
[140] - Quote
On the bounty tracker page: Warning: Invalid argument supplied for foreach() in /var/www/www.evemarketeer.com/mod/missions.php on line 40
And the table is very screwed up now, only numbers where the names should be and all 0's in the amount column.
And it would appear that the totals are still wrong. |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.04 18:46:00 -
[141] - Quote
Ah, awesome, this is the new datadump at work.
It is fixed, and the other problem is fixed as well. I do not show specific rat bounties anymore. This data was not in the SDD anyway, and scraping them from the EVE cache with every database update is going to be obnoxious, and even then it will be inaccurate. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.04 19:09:00 -
[142] - Quote
I think the rat bounties were correct, but the totals are not, that I am sure of. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.04 19:33:00 -
[143] - Quote
Well, the rat bounties were off even when I implemented it, that's why I put the +/- behind the value. It might be that the totals of rat bounties are off too, but I think those are generally correct (unless the API gave me errors).
And I added another feature. You are now able to total your buy and sell orders in the transaction history screen, making for some interesting view of how much you earned on a specific kind of item. It seems PLEXes are one of my major sources of income. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.04 21:31:00 -
[144] - Quote
Callean Drevus wrote:Well, the rat bounties were off even when I implemented it, that's why I put the +/- behind the value. It might be that the totals of rat bounties are off too, but I think those are generally correct (unless the API gave me errors).
And I added another feature. You are now able to total your buy and sell orders in the transaction history screen, making for some interesting view of how much you earned on a specific kind of item. It seems PLEXes are one of my major sources of income. Well something is wrong. Since I rat in null a lot I have gotten to know about how much the rat bounties are. They seemed right on the site, but the image I linked on the previous page shows how much it is off, and the my calculated amount seems right, but your amount was about 20mill too high. If view the last 90 days the number seems a few hundred mill too high, so I know something is wrong. In my opinion, might as well leave the bounties in there as they seemed accurate. Just from looking at your total, it seems about double what it should be.
Also, when will the uploader be fixed? I have been uploading with 1 client for about 2 hours now and just noticed the uploader stopped processing them at 1081. (EMD enabled, EC disabled) |
Scrapyard Bob
EVE University Ivy League
440
|
Posted - 2011.12.05 04:55:00 -
[145] - Quote
http://www.evemarketeer.com/item/info/coolant
I see there is now a "top 5 regions buy/sell" box.
How is that calculate? Is it dropping regions that are more then 7-14 days out of date? |
Scrapyard Bob
EVE University Ivy League
447
|
Posted - 2011.12.06 18:39:00 -
[146] - Quote
Getting strange results from the EMK API feed.
http://www.evemarketeer.com/api/info/37_40_36_11399_38_35_34_39/xml/10000030/sell_lowest5
- This is only returning 7 rows, with the price for Tritanium (34) completely missing. - Also happens in The Forge region. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.06 22:40:00 -
[147] - Quote
Both seem fine to me. Are you still experiencing the issue? I really cannot imagine this happening. The row would have to be deleted from my summary table, and I never actually delete stuff from there unless it's replaced right away.
I'll see if there is any reason this could be happening.
As for the calculation. No, it simply takes the regions that have the most orders, though I imagine it would be a good idea to take the top 5 regions that with the most orders that are not out of date. I'll add that to the calculation.
In regards to bounties. If you put it like that, I'm a whole lot more inclined to think something is wrong with the way I retrieve data from my tables, which could cause a problem. I'll still have to export the bounty prices again though (which I will be doing eventually, but for now I'll first look into the correctness of the totals).
The uploader is a bit of problem right now, as my computer has crashed making me lose the source code of the uploader... which means I'll have to rebuild it. Which is good in the sense that it will probably be without errors, but bad in the sense that it's going to take some time. If you were experiencing no issues with a previous version of the uploader and have disabled EVE Central uploading anyway, you could consider downgrading to a previous version. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.06 22:56:00 -
[148] - Quote
Alright. It's not version control, but have you considered using dropbox? That would save you the hassle in the future for losing things, instant backup and I think it keeps the past 5 versions of a file or something. I only mention that because it is just something you could install and then forget about it. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.06 23:58:00 -
[149] - Quote
I am actually using jungledisk and it is making me very happy. It was the reason I couldn't care less that my harddisks suddenly stopped (that, and all my projects being securely in an offsite version control system, which is a backup by default).
The problem was that it didn't backup my downloads folder (which for the sanity of the backup solution, is a good thing) and the uploader source was in that one folder for some reason, so it's vaporized. But don't worry, as long as EVE Marketdata still has their source I can at least start off a decent basis.
Oh, and I also use dropbox, just not for backups ;) Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.07 00:42:00 -
[150] - Quote
Callean Drevus wrote:I am actually using jungledisk and .............
Alright, thats good. As for the new uploader, maybe put the field for EMK at the top, it is your uploader after all.
Second suggestion, add some way to the uploader to decrease the time. I know its there for people with slower connections so they don't slow it down and miss 1/2 the items, but for me it loads in about 1.5-2 seconds. (Remember, max of 1 JS call per second with the IGB) |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.07 09:06:00 -
[151] - Quote
Actually, the minimum for the market call is 3 seconds, which is why it's at that value by default :P you could make it slower, but you can't make it any faster (unless CCP has changed something without my notice).
The new uploader should follow an alltogether different format of entering that data. Developer/Creator of EVE Marketeer
|
Maeste Madeveda
The Spawning Pool Team Liquid
0
|
Posted - 2011.12.08 12:40:00 -
[152] - Quote
Hello Callean,
I was starting to use your website because it the shiniest one and I love shiny stuff. However as I wished to register, the site denied me the entry without sending me back any error message. Am I not good enough for your upload club ? :(
Win 7 x64 Chrome 13.0.767.1 |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.12.08 19:56:00 -
[153] - Quote
I've just noticed that I'm receiving a lot of:
Debug: upload: uploading to http://www.evemarketeer.com/api/upload Debug: upload: uploader.send = 200, OK Debug: upload: response=0: The upload cannot come from the future Error: Upload unknown problem: 0: The upload cannot come from the future Debug: returned from upload function
I'm definitely not doing anything that allows me to see future market values, but I sure wish that I could. :)
EDIT: BTW, it doesn't happen for all items. There doesn't seem to be a pattern to exactly which items as far as I can tell. Some example item ID's that caused it in the Verge Vendor region, in case it helps:
20843 13239 30078 32108 28471 21238 22577 |
Zaepho
Phoenix Propulsion Labs RED Citizens
6
|
Posted - 2011.12.08 20:08:00 -
[154] - Quote
Turhan Bey wrote:I've just noticed that I'm receiving a lot of: Debug: upload: uploading to http://www.evemarketeer.com/api/uploadDebug: upload: uploader.send = 200, OK Debug: upload: response=0: The upload cannot come from the future Error: Upload unknown problem: 0: The upload cannot come from the future Debug: returned from upload function
Possibly clock skew by a few seconds or something along those lines? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.08 22:21:00 -
[155] - Quote
Maeste, I've just send you an EVE Mail, registration should be working again. It was indeed broken for a few days, but people were very fast to point it out :)
The only reason for the values to come from the future is indeed if your system clock is by some reason ahead of the EVE clock. This has been built in mostly to prevent people from sending illegal data that will always be seen as the most recent. I might relax the requirement a bit so as not to fall over a few seconds difference :) Developer/Creator of EVE Marketeer
|
Zaepho
Phoenix Propulsion Labs RED Citizens
6
|
Posted - 2011.12.08 22:49:00 -
[156] - Quote
Callean Drevus wrote: The only reason for the values to come from the future is indeed if your system clock is by some reason ahead of the EVE clock. This has been built in mostly to prevent people from sending illegal data that will always be seen as the most recent. I might relax the requirement a bit so as not to fall over a few seconds difference :)
I don't suppose you have a log of these skew values to make an intelligent decision of how far off is commonly acceptable? I would think about 5 or 10 seconds should be MORE than plenty but I'm confident my machines talk to a decent NTP server on a regular basis :) |
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.12.08 23:38:00 -
[157] - Quote
Likewise, I use NTP so I shouldn't be much more than a second or two different. Unless of course the EVE servers don't and they are drifting. I agree that 10s or so should be sufficient to deter false data but still account for sync issues. |
Grimdahl
The Leather Knights Sev3rance
3
|
Posted - 2011.12.09 15:41:00 -
[158] - Quote
Is there any order filtering applied eg. drop order with highest price?
Reason for my question is: I compared market order data from Eve-Central and Eve-Marketeer and found 1 order missing (Minmatar shuttle : sell order 888.888.888 ISK in Jita).
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.09 23:22:00 -
[159] - Quote
A lot of order filtering is applied. It drops all outrageous orders. Developer/Creator of EVE Marketeer
|
Turhan Bey
EVE University Ivy League
0
|
Posted - 2011.12.10 20:35:00 -
[160] - Quote
Minor request: Upon opening the EMK Uploader window, please don't make the "EVE Marketdata Upload ID" field selected by default. Any accidental typing wipes out the field.
I've gotten into the habit of clicking in the Log field after every launch. But it would be nice if that were unnecessary. |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.10 20:48:00 -
[161] - Quote
Rest assured :P the new uploader won't do any such thing ;) Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
452
|
Posted - 2011.12.10 23:18:00 -
[162] - Quote
Zaepho wrote: I don't suppose you have a log of these skew values to make an intelligent decision of how far off is commonly acceptable? I would think about 5 or 10 seconds should be MORE than plenty but I'm confident my machines talk to a decent NTP server on a regular basis :)
Well, the "windows time server" a.k.a. "time.windows.com" is commonly off by a bit. And older WinXP machines will only sync with the server about once a week unless you play with the registry or belong to a Windows domain. So 2-3 seconds is not uncommon on normal Windows boxes.
So I'd say 5 seconds should be fine, unless someone's machine is lagged in which case maybe you have to go with 10 seconds.
|
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.11 00:46:00 -
[163] - Quote
I'd allow 10 lag and 5 ahead and it shouldn't be a problem after that. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.11 09:24:00 -
[164] - Quote
Well, atm I only check whether a message comes from the future. I do not really mind getting messages from anywhere in the past. That is mostly due to the fact that uploads might end up in a queue and be recieved a lot later than they were generated, and I do not wish to exclude those.
Oh, the new uploader will probably be using a bit more memory than the old one, a function of how I deal with threaded uploading and still clearing the cache quickly. As long as you upload decently fast (and do not have 200 endpoints) it shouldn't be problematic though ('m thinking a few MB's of ram). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
452
|
Posted - 2011.12.11 15:04:00 -
[165] - Quote
The existing EVEMarketeer uploader reports as about 47MB used (peak) in Win7 64bit. (The EMD uploader is very similar size.)
So unless you blow that up to more then 80-ish, you should be fine. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.11 17:45:00 -
[166] - Quote
I present to you, the unified uploader: http://www.evemarketeer.com/downloads
You can now add any number of endpoints, have EVE Central lagging behind all other endpoints without any trouble and it be generally awesome and out of your way.
Currently only downloadable as a zip archive, but it'll change once I've worked out all the bugs. I invite you to try it out! Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.12 07:39:00 -
[167] - Quote
Quick bug report here for you Warning: Invalid argument supplied for foreach() in /var/www/www.evemarketeer.com/mod/item.php on line 88 Market statistics and information for New Eden Catchable fatal error: Object of class yapeal could not be converted to string in /var/www/www.evemarketeer.com/var/dc9f76fa4c888904204769a29d0ad5f2.d15.php on line 15 Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.12 07:40:00 -
[168] - Quote
Callean Drevus wrote:I present to you, the unified uploader: http://www.evemarketeer.com/downloadsYou can now add any number of endpoints, have EVE Central lagging behind all other endpoints without any trouble and it be generally awesome and out of your way. Currently only downloadable as a zip archive, but it'll change once I've worked out all the bugs. I invite you to try it out!
Looks nice.
1) You did forget the 's' in universal under data types when adding a site.
2) [Edit] A restart of the program fixed this, but still not sure why it happened the first time.[/edit] Also, you might want to fix the cache folder. It starts out with the windows username of Bart, and autodetect isn't changing it on my desktop, though it does fix it on my laptop. Maybe have it autodetect on first startup.
Quote:Traceback (most recent call last): File "UploaderFrame.pyo", line 303, in installationAutodetectButtonClick AttributeError: Installation instance has no attribute 'folder_base'
3) Maybe add a upload log like before so we can tell if its working. if it does have one, then it isnt working for me. |
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.12 08:06:00 -
[169] - Quote
On another note Not sure if the new uploader is doing anything it doesn't seem to be uploading anything Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.12 08:21:00 -
[170] - Quote
Dragonaire wrote:On another note Not sure if the new uploader is doing anything it doesn't seem to be uploading anything Odd, my upload history on the site is completely empty. |
|
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.12 16:07:00 -
[171] - Quote
Mine seems to have done some uploads now but only after disabling the other sites and a couple restarts Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.12 17:57:00 -
[172] - Quote
Hmm, the only problem I am aware of is having to restart after autodetecting for the shizzle to work. I'll see wha I can do about these current troubles. The upload log should be working :P if it isn't showing anything, it isn't uploading. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.12 20:11:00 -
[173] - Quote
Callean Drevus wrote:Hmm, the only problem I am aware of is having to restart after autodetecting for the shizzle to work. I'll see wha I can do about these current troubles. The upload log should be working :P if it isn't showing anything, it isn't uploading. The new uploader isn't uploading anything on either of my computers. |
Scrapyard Bob
EVE University Ivy League
452
|
Posted - 2011.12.13 04:12:00 -
[174] - Quote
Will try this when you release the new version (I promise). |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.13 06:09:00 -
[175] - Quote
Counting is either off again or still off for ratting. In the past 24 hours I have made between 50-70 mill, not 292,519,637.50 as the site says.
Edit: The correct count is 105 mill, though it was actually in the range I said above due to having another person there. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
38
|
Posted - 2011.12.13 17:12:00 -
[176] - Quote
Counting is still off Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.13 17:56:00 -
[177] - Quote
I've just been thinking about something and thought I'd ask here. How do you come up with the list of stuff in the general list for the scans? Do you just do it randomly or by market group or are you trying to get the best coverage for things you don't have data on yet? Or have you not really thought about how to optimize things? Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.13 18:41:00 -
[178] - Quote
The list is sorted based on what was last uploaded (in the region you reside in). That means the most outdated items are uploaded first. I'm not entirely sure whether it also does this for subsets.
I've also uploaded a new version of the unified uploader. It hasn't really changed much, but it should autodetect when you have no EVE installations defined and automatically search for them.
I'm also sorry to say that your totals have been fixed Vrelk I was counting the total wallet journal amount for every rat you killed Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
452
|
Posted - 2011.12.13 19:22:00 -
[179] - Quote
v0.2 of the unified uploader, it starts on Win7 64bit, creates an icon in the systray, but then never does anything. The log file contains:
Traceback (most recent call last): File "emkUploader.py", line 19, in module File "wx\_core.pyo", line 7981, in __init__ File "wx\_core.pyo", line 7555, in _BootstrapApp File "emkUploader.py", line 11, in OnInit File "UploaderFrame.pyo", line 70, in __init__ File "UploaderFrame.pyo", line 107, in initialize File "UploaderFrame.pyo", line 305, in installationAutodetectButtonClick AttributeError: Installation instance has no attribute 'folder_base' |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.13 21:15:00 -
[180] - Quote
Hmm, will take a look.
EDIT: Fixed version uploaded Developer/Creator of EVE Marketeer
|
|
Scrapyard Bob
EVE University Ivy League
453
|
Posted - 2011.12.14 00:24:00 -
[181] - Quote
Searching for "plex" returns a page with a bunch of items that contains the word "plex". Which is all well and good.
But you can't click on the line that says "30 Day Pilot's License Extension (PLEX".
My guess is that it breaks due to the single-quote inside of the item's name. |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.14 02:15:00 -
[182] - Quote
I will have to say, the mission page is completely confusing me. Yesterday it was a lot higher than it should have been, today it is spot on.
1) The uploader now uploads 2) it uploads before you enter a upload key 3) It stops responding when you try to close it 4) It says version 1.0 when it should say 0.2 |
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.14 05:45:00 -
[183] - Quote
Quote:2) it uploads before you enter a upload key But with error message about not having key.
Quote:4) It says version 1.0 when it should say 0.2 He beat me to it I noticed it on version 0.1 also but figured you'd notice and fix it on next update but seems you didn't.
Quote:3) It stops responding when you try to close it Edit I have notice this now and had to have Windows kill it because it wouldn't close.I have also notice if you stop scanning for a couple minutes and then restart the scan in Eve you have to kill and restart it to get it going again.
Another issue I've noticed is after it has been running for a while it seems to start refreshing the upload tab and goes to the top than jumps down to the end again I believe to add the next item but once again goes back to the top so you can't tell if it is still working or not so most of the time I end up just restarting it since I have no idea if it is still working or has locked up. This happened some with 0.1 but seems worse with 0.2 in that it seems to do it sooner. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Scrapyard Bob
EVE University Ivy League
455
|
Posted - 2011.12.14 17:34:00 -
[184] - Quote
log entries for the 0.2b client
Exception in thread EVE Marketeer: Traceback (most recent call last): File "threading.pyo", line 552, in __bootstrap_inner File "Classes.pyo", line 381, in run File "UploaderFrame.pyo", line 94, in log File "wx\_controls.pyo", line 1850, in AppendText PyAssertionError: C++ assertion "m_count == -1 || m_count == -2" failed at ..\..\src\msw\textctrl.cpp(140) in UpdatesCountFilter::UpdatesCountFilter(): wrong initial m_updatesCount value
Also, I have to double-click the system tray icon to get it back to the foreground, in the past tools I only had to single-click the system tray icon to bring it forward.
The "endpoints" tab (or status tab) should also keep track of:
- How many market orders have been uploaded to each endpoint since the tool started running. - The average time, in seconds, that it took to give each upload to the endpoint server (from "I have something for you" to "okay, it's accepted" to gauge responsiveness). - How big the upload queue is for that particular endpoint. - Age (in seconds) of the oldest item in the endpoint's queue. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.14 18:10:00 -
[185] - Quote
Controller Vrelk wrote:I will have to say, the mission page is completely confusing me. Yesterday it was a lot higher than it should have been, today it is spot on.
That's because I fixed it :) I more or less wrote than in a previous post, but probably not in a way that was clear enough.
I will try to fix most of these issues and upload a new uploader. Thank you for being my test subjects
Quote:My guess is that it breaks due to the single-quote inside of the item's name.
Correct. Fixed. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.14 20:17:00 -
[186] - Quote
Okey, the new uploader 0.3 is released.
1) It should hopefully still upload 2) It should only upload before an upload key is entered for EVE Central, it does show the message of not being able to upload with each file it parses though. 3) This happened for some reason if the uploader still had uploads in the queue when you closed it. I've now disabled closing (and I'm showing a message telling you so) while the uploader is still uploading. Not ideal, and I hope to fix that later, but at least you now know what is happening. 4) Ah, well, I of course knew about it, but opening my interface editing program was just a bit too much trouble.
5) The "wrong initial m_updatesCount value" exception should be fixed 6) The "endpoints" tab now keeps track of all mentioned things, except for the time the oldest upload was entered, this was impossible due to the way the queues are made. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
455
|
Posted - 2011.12.15 04:11:00 -
[187] - Quote
0.3 is much nicer - it forgot my EMK key that I had entered in version 0.2, but that was no big deal. The stats are now much more informative and give a feel for "is it working" better then watching the list scroll by. |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.15 04:39:00 -
[188] - Quote
I would recommend auto-generating the config file. That way when it is extracted, you don't overwrite the old config file that contains the keys, although I was paying attention not to overwrite it, some people may. |
Scrapyard Bob
EVE University Ivy League
455
|
Posted - 2011.12.15 04:43:00 -
[189] - Quote
Controller Vrelk wrote:I would recommend auto-generating the config file. That way when it is extracted, you don't overwrite the old config file that contains the keys, although I was paying attention not to overwrite it, some people may.
Ah, that's what I had done wrong - I deleted the contents of my EMK upload program folder, which contained the XML file, before upgrading to 0.3.
(It was able to pull the EMD key from my EMD upload client, so that's why it didn't forget the EMD key.) |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.15 04:47:00 -
[190] - Quote
The uploader (0.3) isn't uploading anything for me. The queue size kept increasing until it hit 50 and then it stopped, but the uploads is still 1, which it did as soon as I opened it. It appears to be queuing everything and not uploading. I have a key entered for both EMK and EMD, and 0 for EC.
|
|
Scrapyard Bob
EVE University Ivy League
455
|
Posted - 2011.12.15 04:55:00 -
[191] - Quote
Controller Vrelk wrote:The uploader (0.3) isn't uploading anything for me. The queue size kept increasing until it hit 50 and then it stopped, but the uploads is still 1, which it did as soon as I opened it. It appears to be queuing everything and not uploading. I have a key entered for both EMK and EMD, and 0 for EC.
Mine uploaded about 450-550 to each of the (3) sites, then stopped working.
Attempting to close the uploader resulted in the following error:
Exception in thread Thread-4: Traceback (most recent call last): File "threading.pyo", line 552, in __bootstrap_inner File "Classes.pyo", line 303, in run File "Classes.pyo", line 329, in addNewFiles IOError: [Errno 13] Permission denied: 'C:\\Users\\ScrapyardBob\\AppData\\Local\\CCP\\EVE\\c_program_files_(x86)_ccp_eve31024_tranquility\\cache\\MachoNet\\87.237.38.200\\301\\CachedMethodCalls\\8c18.cache'
Now, that could possibly be because I still have the EMD's uploader running as a fail safe against the EMK uploader hanging / stopping.
Restarted it and it picks right up and processes again. Will watch and see whether it hangs at around the same point. |
Scrapyard Bob
EVE University Ivy League
455
|
Posted - 2011.12.15 05:03:00 -
[192] - Quote
Controller Vrelk wrote:The uploader (0.3) isn't uploading anything for me. The queue size kept increasing until it hit 50 and then it stopped, but the uploads is still 1, which it did as soon as I opened it. It appears to be queuing everything and not uploading. I have a key entered for both EMK and EMD, and 0 for EC.
I hit this bug too. After a few minutes of working fine (a few hundred uploads) it decided to stop uploading on all (3) threads and is just sitting there with the queue column at 50 and the current time column constantly counting upwards.
(The EMD uploader is still running strong, so EMD is up.)
While the "Wait until uploads are finished" message is nice, I'm wondering if you can eventually add a button to that dialog that says "Go ahead and exit" beside the "Oops, don't exit" button. |
Scrapyard Bob
EVE University Ivy League
455
|
Posted - 2011.12.15 05:07:00 -
[193] - Quote
Seems like the issue always happens after:
EVE Central: Upload unknown problem: EVE Central: Uploaded ##### in ######## EVE Central: None
The log has a few entries of:
Exception in thread EVE Central: Traceback (most recent call last): File "threading.pyo", line 552, in __bootstrap_inner File "Classes.pyo", line 399, in run AttributeError: 'NoneType' object has no attribute 'find'
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.15 18:59:00 -
[194] - Quote
Config file will eventually just be placed in appdata (or home directory, for unix users).
It also seems that the uploader encounters some errors, quits processing the threads, and then continues running without end, without actually doing something. Thanks to your clear error messages I know exactly what is happening and will probably be able to fix it tonight.
I actually really want the "just go and exit this damn thing" button, but somehow, python does not allow it. Though it seems I've just found a function that does, guess I just wasn't able to find it yesterday. Thanks again for helping point out the issues! Developer/Creator of EVE Marketeer
|
Major Thrasher
Native Freshfood Minmatar Republic
0
|
Posted - 2011.12.15 20:32:00 -
[195] - Quote
could we have filter options on the stats pages in trade information. currently its just full of minerals and junk.
drop down menu's with full selectable lists would be awsome also.
being able to see the info i want rather than everything i don't would be much more useful IMO.
another suggestiong is that the upload uploads info that hasn't been done before. for example when i start a full scan smartbomb is always first even if i just scanned and uploaded it, then restarted pc. items should update in order of longest time since upload. similar to evemarket data. |
Scrapyard Bob
EVE University Ivy League
456
|
Posted - 2011.12.16 05:42:00 -
[196] - Quote
Tool seems to be working better - but EVEMarketeer things that some/most of my uploads are coming from the future. (The other 2 endpoints are happy with the upload data, so I'm guessing that EMK is being too picky or has a clock that has drifted.) |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.16 05:55:00 -
[197] - Quote
Oh, I love how these forums don't post things... I posted this a few hours only to find out now that it never posted it.
Anyway.
What is this? http://www.evemarketeer.com/item/info/Gallente_Thanatos_Carrier I found that while searching for "Thanatos" |
Scrapyard Bob
EVE University Ivy League
456
|
Posted - 2011.12.16 12:41:00 -
[198] - Quote
The crashes are still happening after a random amount of time.
Problem signature: Problem Event Name:APPCRASH Application Name:unifieduploader.exe Application Version:0.1.0.0 Application Timestamp:49180193 Fault Module Name:KERNELBASE.dll Fault Module Version:6.1.7601.17651 Fault Module Timestamp:4e211319 Exception Code:80000003 Exception Offset:0001280c OS Version:6.1.7601.2.1.0.256.48 Locale ID:1033 Additional Information 1:0a9e Additional Information 2:0a9e372d3b4ad19135b953a78882e789 Additional Information 3:0a9e Additional Information 4:0a9e372d3b4ad19135b953a78882e789
In this case, it had uploaded 3900 to EMK, 5751 to EMD, 6671 to EC. No log file was generated, nothing unusual in what can be seen of the Status window. Nothing unusual in the status columns, no full queues. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.16 19:56:00 -
[199] - Quote
Interesting :) this one must be from the new datadump :P
EDIT: And I'm on the uploader problem. Will try uploading for a long time myself this evening.
Oh, and you might've noticed the "EVE Central" user boosting itself to the top of the upload list. This is not a user that is pretending to be something he's not, it's just the one I've added to keep track of the quality and number of EVE Central uploads I retrieve.
I'll have to exclude him from the homepage record though :) Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.16 21:01:00 -
[200] - Quote
Major Thrasher wrote:another suggestiong is that the upload uploads info that hasn't been done before. for example when i start a full scan smartbomb is always first even if i just scanned and uploaded it, then restarted pc. items should update in order of longest time since upload. similar to evemarket data.
They, are.
EVE at the moment is not being nice to us marketeers in that it doesn't seem to be writing a cache file when there are no orders. Which explains why I get not upload, which explains why you always see the smartbomb first. If I receive no uploads, I cannot change the order of the list (I could change it based on what is shown, but you never know whether that is actually uploaded). Developer/Creator of EVE Marketeer
|
|
Scrapyard Bob
EVE University Ivy League
456
|
Posted - 2011.12.16 21:48:00 -
[201] - Quote
Yeah, no cache file = nothing to grab.
Some possibly cheap fixes:
- When a type ID + region pair is handed out, update the record with a "last handed out" time stamp.
- Keep track of the last time that a particular typeID+region pair has gotten data. (Which you already do.)
1) With the presence of a "last handed out" time stamp, you could pull a list of stuff that needs checking, but exclude anything that was handed out in the previous N minutes (30 minutes might be good). That way if you have (2) clients checking a particular region, both will get slightly different lists to work from. This method does start to break down once you have (20+) clients all checking the same region at the same time, but that's a good problem to have.
2) In addition you might choose to add a few minutes/hours to the "last handed out" time stamp to delay checking them again. For example, if the "last update time" > 7 days, then push the "last handed out" time stamp 2.56 hours into the future. If the "last update time" > 14 days, then push the "last handed out time" 3.23 hours into the future.
"last handed out" = (now) + cube root of (# hours since last update / 10)
cube root (24/10) = 1.34 cube root (168/10) = 2.56 cube root (336/10) = 3.23 cube root (720/10) = 4.16 cube root(2160/10) = 6.00
(You may decide to divide by 30 or 50 instead of 10, to shorten up the last handed out bump.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.16 21:59:00 -
[202] - Quote
Yeah, I had been considering those options too (not with a separate timestamp, but just bumping up the last uploaded one). But I would first like to see that this is really a problem before trying to fix it.
Though it is a good idea for solving the problem of multiple uploaders per region, so I'll probably implement this anyway.
I'm not completely sure of what you hope to achieve by bumping the 'last handed out' time into the future for uploads for which it has been a very long time since the last upload. Those are generally the ones I really want to check , since I want that data. Can you explain your thoughts to me?
EDIT: Okey, never mind my first paragraph, this is a good idea regardless of what the reason for not repeating requests is. I'm getting tired of uploading Zaino Deadeye's that do not exist. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
456
|
Posted - 2011.12.16 23:14:00 -
[203] - Quote
It's mostly to deal with the issue of things like the Deadeyes where if you haven't seen it in a month or three months, then it's probably not worth checking for near as often. Depending on what factors you use as the divisor, you'd still be handing it out a few times per day, just not every single time. As long as the different sites (such as EMD/EC) do their algorithms slightly differently, it shouldn't be a weakness.
And there's still the issue where the EMK scanner page "breaks" after 2-3 hours and stops opening up the market window, even though the page is still refreshing. My guess is that you're up against some sort of bug in the web browser, or you're generating requests slightly faster then the 3 second rule. I'm still using eve-marketdata's market scanner page whenever I do my overnight updates. |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.12.17 00:21:00 -
[204] - Quote
I was noticing this a lot when uploading items in Fountain. When I had two clients going, one in Jita, and another in fountain, over a 2 hour period the Jita client seemed to be uploading 10x the amount. I noticed that a lot of the time, BPO's not seeded in regions were taking up a ton of time. Same with those rarely used implants. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.17 09:11:00 -
[205] - Quote
Yes, I noticed the same BPO trouble yesterday. And due always trying the things that have been uploaded last, the uploader just jams when there are 50 items that will never be uploaded at all. Today I'll take a look at the uploading page and ways in which it can be improved. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.17 15:30:00 -
[206] - Quote
I've uploaded a new version of the unifieduploader. The only thing this changes is that it has function built in to log what is going wrong with the uploader threads. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
456
|
Posted - 2011.12.17 21:50:00 -
[207] - Quote
It tends to crash sooner if you leave the window open then if you minimize it (not fully verified, but a working hypothesis).
EVE Marketeer endpoint log:
Traceback (most recent call last): File "Classes.pyo", line 423, in run File "UploaderFrame.pyo", line 119, in log File "wx\_controls.pyo", line 1850, in AppendText PyAssertionError: C++ assertion "m_count == -1 || m_count == -2" failed at ..\..\src\msw\textctrl.cpp(140) in UpdatesCountFilter::UpdatesCountFilter(): wrong initial m_updatesCount value
EMD endpoint log:
Traceback (most recent call last): File "Classes.pyo", line 423, in run File "UploaderFrame.pyo", line 122, in log File "wx\_controls.pyo", line 1818, in Remove PyAssertionError: C++ assertion "m_count == -1 || m_count == -2" failed at ..\..\src\msw\textctrl.cpp(140) in UpdatesCountFilter::UpdatesCountFilter(): wrong initial m_updatesCount value
EC endpoint log:
Traceback (most recent call last): File "Classes.pyo", line 423, in run File "UploaderFrame.pyo", line 119, in log File "wx\_controls.pyo", line 1850, in AppendText PyAssertionError: C++ assertion "m_count == -1 || m_count == -2" failed at ..\..\src\msw\textctrl.cpp(140) in UpdatesCountFilter::UpdatesCountFilter(): wrong initial m_updatesCount value
file reader log - nothing. |
Scrapyard Bob
EVE University Ivy League
456
|
Posted - 2011.12.17 22:12:00 -
[208] - Quote
Scrapyard Bob wrote:It tends to crash sooner if you leave the window open then if you minimize it (not fully verified, but a working hypothesis).
Working hypothesis disproven, it still crashes frequently when minimized to the system tray.
My guess is that once you get too many of the following - it just ups and dies:
Traceback (most recent call last): File "Classes.pyo", line 423, in run File "UploaderFrame.pyo", line 122, in log File "wx\_controls.pyo", line 1818, in Remove PyAssertionError: C++ assertion "m_count == -1 || m_count == -2" failed at ..\..\src\msw\textctrl.cpp(140) in UpdatesCountFilter::UpdatesCountFilter(): wrong initial m_updatesCount value
The log files are full of the above message.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.17 22:49:00 -
[209] - Quote
I know, I've been having these messages for the better part of the day, and my uploader still hasn't crashed...
What does it do? Does it simply hang? Or does it seem to not do anything anymore (but still being responsive)? Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. PURgE Alliance
23
|
Posted - 2011.12.17 23:11:00 -
[210] - Quote
I haven't try the latest version but on the older version it still seemed to respond just stop uploading anything etc.
On another note I'm not sure if you saw the Eve-mail I sent you but please read it as something that I fixed in Yapeal could be causing some very high load issue for your site plus there been several changes to the APIs you might want to let people see/use Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
39
|
Posted - 2011.12.17 23:36:00 -
[211] - Quote
Ah, yes, I've seen it, and it's been lodged firmly in the back of my mind since then :P
As to the not uploading anymore issue, that is probably (as I've found after a day of uploading) because there is nothing for the uploader to upload anymore. After cycling trought the entire market, the uploader starts anew, but if the same EVE client is running, and nothing has changed in the market information, the cache file isn't rewritten, thus there is nothing for the uploader to do. To check this, your best bet would be to manually lookup Tritanium and see if the uploader responds to that, if it doesn't, try buying one unit of Trit and see if it works then (I'll refund it ). If it doesn't, then something is very certainly broken.
Of course, it could be that it suddenly starts uploading again when you restart the upload client, which is a practical guarantee of something going wrong. Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. PURgE Alliance
23
|
Posted - 2011.12.17 23:55:00 -
[212] - Quote
Quote:Of course, it could be that it suddenly starts uploading again when you restart the upload client, which is a practical guarantee of something going wrong. That what I had happening also if I finish doing say the Tech II BPOs then decide I only had time also do something like Minerals it would not restart uploading anything until restarted. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Scrapyard Bob
EVE University Ivy League
457
|
Posted - 2011.12.18 07:52:00 -
[213] - Quote
Callean Drevus wrote:I know, I've been having these messages for the better part of the day, and my uploader still hasn't crashed...
What does it do? Does it simply hang? Or does it seem to not do anything anymore (but still being responsive)?
It just keeps crashing on Win7 64bit - minimized, not minimized. I'm running 4 clients at it at the same time, so it's under a bit of stress. But it never uses much CPU and stays with a peak working set of about 39MB. It can run for an hour and not crash, or crash after 5 minutes, no set pattern - which means it's probably reacting to either a particular item, or a network condition when talking to the end point servers.
I can't tell whether the errors are related to the crash or just background noise, but it's the only thing showing up in the log files.
|
Dragonaire
Corax. PURgE Alliance
23
|
Posted - 2011.12.18 08:33:00 -
[214] - Quote
I'm also running on Windows 7 64 bit if that helps and I can't say I've had it crash really just stop uploading etc. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Scrapyard Bob
EVE University Ivy League
457
|
Posted - 2011.12.18 09:09:00 -
[215] - Quote
Not sure then - will do a fresh install folder and see what happens.
When I do my uploads overnight, I run multiple clients, all from the same cache directory. Sometimes 2 in the same region.
The new upload page does seem to be much better behaved and doesn't stop working like the old one did. (Nice "Christmas" theme *grin*) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
40
|
Posted - 2011.12.18 09:22:00 -
[216] - Quote
Seems like I'll have to grab my old and dusty two alts then to do a bit of stress testing on the uploader, it must be related to uploading from multiple accounts at the same time.
And thanks, I very much like the theme too Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
457
|
Posted - 2011.12.18 18:26:00 -
[217] - Quote
Mine paused uploading today, new error message:
Traceback (most recent call last): File "Classes.pyo", line 427, in run AttributeError: 'NoneType' object has no attribute 'find'
Also, I finally got the EMK uploader to fight over a cache file with the EMD uploader and got this in the file reader log file:
Traceback (most recent call last): File "Classes.pyo", line 310, in run File "Classes.pyo", line 339, in addNewFiles IOError: [Errno 13] Permission denied: 'C:\\Users\\ScrapyardBob\\AppData\\Local\\CCP\\EVE\\c_program_files_(x86)_ccp_eve31024_tranquility\\cache\\MachoNet\\87.237.38.200\\301\\CachedMethodCalls\\7d4d.cache'
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.12.19 08:04:00 -
[218] - Quote
oh god, what did you do? the market page just keeps giving me gypsy and noble implants to attempt to upload, except there's none on the market, hence no uploads...... |
Scrapyard Bob
EVE University Ivy League
459
|
Posted - 2011.12.19 15:10:00 -
[219] - Quote
Domi Etymology wrote:oh god, what did you do? the market page just keeps giving me gypsy and noble implants to attempt to upload, except there's none on the market, hence no uploads......
Looks like it might need some tuning. (I often switch between using eve-marketdata's scanner page, eve-central's page, and EMK's page through the week.)
Thinks about the numbers... There are 6922 possible market items, 3.5 seconds per item. In an ideal world, that means you can scan the market with a single alt in the space of 6.73 hours (5.77 hours w/ the 3 sec version, but that breaks). If we have (2) bits of information (last upload, last handed out), then we need to penalize items that haven't updated recently and which have also been handed out recently. Logic dictates that you should create a 3rd field to handle this desired result and call it "next possible queue time", which makes the market scanner's page a lot simpler.
LUT = last upload time stamp (defaults to something like the start of the epoch, or something like January 2000) LHO = last handed out time NQT = next queue time (defaults to start of epoch)
NQT should be updated when either an item is handed out, or a successful upload arrives.
LHO event -> set NQT to current time + cuberoot(# hours since LUT / 25) + random(1800 seconds) - So as items are handed out, they get pushed down deeper into the stack, which will bubble other items up to the top. - Adds a bit of randomness to move things up/down by a maximum 15 minutes - The "25" divisor gives good results for items which are only seen once a year (bumps them by 7.05 hours)
LUT event (successful upload) -> set NQT to current time + 8 hours + random(4 hours) - When an item is successfully uploaded, it needs to move down towards the bottom of the stack - Since the 6922*3.5 sec = 6.73 hours, the magic number is 8 hours plus up to 4 hours
Now for the reason that we use a NQT field instead of looking at the LHO/LUT fields when figuring out what the market scanner should hand out next. When the market scanner page needs to pull 20 items to hand out, it should query the item field, sorting on NQT in ascending order, top 20 items. This way, you always get 20 items, even if the NQT values are hours/days into the future.
|
Scrapyard Bob
EVE University Ivy League
459
|
Posted - 2011.12.19 17:21:00 -
[220] - Quote
Callean Drevus wrote:Seems like I'll have to grab my old and dusty two alts then to do a bit of stress testing on the uploader, it must be related to uploading from multiple accounts at the same time.
I can trigger it by using the EMD market scanner page, using just two accounts - even if the accounts don't share a cache directory. More simultaneous accounts just triggers the bug faster / more often. |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
41
|
Posted - 2011.12.20 20:26:00 -
[221] - Quote
Ok, now testing scanning with multiple accounts. At the very least I've found the error that was producing the EVE central errors. Now I'm seeing the EVE Marketdata timeout errors (upload within 15 minutes), so I can do something about those I hope. What I fail to understand is how my fifo queue can upload some orders correctly, only to fail on the next one...
UPDATE: Ok, rewriting the filereader led me to probably find what was causing the uploader to hang. It found a lot of files which it wasn't able to do anything with. But these files kept heaping up, because it didn't delete them, leading to what were probably immense times for scanning files. In any case, I'll keep scanning and see what happens.
UPDATE 2: The uploader get's better and better. It's now uploading into the future I've got more uploads than file reads . I guess something is still going wrong with threading. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
41
|
Posted - 2011.12.20 22:48:00 -
[222] - Quote
I've uploaded a new version of the uploader. I hope this one improves stability, but it's still a bit stabbing in the dark, since I've yet to receive the errors myself.
And: Quote:oh god, what did you do? the market page just keeps giving me gypsy and noble implants to attempt to upload, except there's none on the market, hence no uploads......
I'm sorry, I was sorting in the wrong direction Developer/Creator of EVE Marketeer
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2011.12.21 03:37:00 -
[223] - Quote
serves you right for writing code while standing on your head.
The new images showing the item list of what's being scanned/uploaded are sweet :) |
Scrapyard Bob
EVE University Ivy League
460
|
Posted - 2011.12.21 15:36:00 -
[224] - Quote
I'll beat on the 0.4 version of the uploader today.
Also, the market scanner page seems a bit odd still when viewed in the in-game browser. Sometimes the icons go down the left side of the page, other times they just go in a line across. Could be browser width issues. (Will try and figure out more later.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
42
|
Posted - 2011.12.21 16:00:00 -
[225] - Quote
That might be caching issues, but I'm not sure. Only time I've seen something like that was the first time I loaded the page. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.21 19:30:00 -
[226] - Quote
Domi Etymology wrote:serves you right for writing code while standing on your head.
The new images showing the item list of what's being scanned/uploaded are sweet :) Yea, just noticed them. Let me guess, I gave him the idea. |
Scrapyard Bob
EVE University Ivy League
461
|
Posted - 2011.12.21 20:13:00 -
[227] - Quote
Callean Drevus wrote:That might be caching issues, but I'm not sure. Only time I've seen something like that was the first time I loaded the page.
Yes, possibly cache issue, the site is now behaving on the clients where I'm running them at 1024x768 (so the browser window is only around 850-900 wide.
So far 0.4 of the uploader is passing with flying colors at keeping up with (3) simultaneous market scan clients running. Memory peaked out at around 62.4MB, CPU usage has been negligible. I've uploaded about 14k to each of the (3) endpoints in today's session.
I used EMD's scanner site for most of the day, but will switch over to using the EMK's scanner tomorrow.
Other small notes:
- The XML/JSON/CSV links on individual item pages all point at the "xml" version of the URL
- The location of the regional drop down still feels too far out of place in the upper right, maybe it should be where the item's icon is on an individual item's page. This would probably force you to move the item's icon to the left side of that area.
- Price history graphs look like they broke 3 weeks back and haven't updated.
- It would be nice if I'm on a particular item's page (say Tritanium) that I could click on a link next to the item's name (such as "Minerals") which would take me to a market group summary page for that particular item.
- Longer term wish, being able to see an entire market group (i.e. "Minerals"), with the individual items down the left and the top 5 market regions identified as column headers with the 5% buy/sell prices as the cell contents. Ignoring issues with color blindness, if values where the individual region is 5% above/below the average of the top-5 market price was colored green/red, that might be useful. Prices listed would only need to have about 4-5 decimal places of precision. This means you could switch to "k", "M", "B", "T" suffixes to trim digits off the tail end. The average price in the entire grid should control which suffix to use (up to 50,000 use no divisor, up to 1M use the 'k', up to 1B use the 'M', up to 1T use the 'B'). So you would get numbers like 4385.20, 43850.20, 438.52k, 4.3852M, 43.852M, 438.52M, 4.3852B, 43.852B, 438.52B, 4.3852T, etc.
|
Scrapyard Bob
EVE University Ivy League
461
|
Posted - 2011.12.22 18:26:00 -
[228] - Quote
0.4 is still working fine, even with multiple clients running (same region, same cache folder, etc.). Zero errors in the log files now. The market scanner page also continues to perform well.
Minor requests for the 0.4 client:
- Add a "restore" option to the right-click menu of the system tray icon. - When I double-click on the sys tray icon, it always comes back to the foreground in a minimized state. - Upload keys might need to be moved out of "config.ini" and into a "user.ini".
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
42
|
Posted - 2011.12.22 22:02:00 -
[229] - Quote
Yes, very feasible requests all. I'll see when I'll be able to incorporate them into the codebase. Currently incredibly busy with RL and being sick again.
UPDATE: Also very happy to hear that the uploader is still performing perfectly :)
UPDATE 2: Also, PERL and especially CPAN are absolutely terrible... Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. PURgE Alliance
25
|
Posted - 2011.12.23 17:24:00 -
[230] - Quote
Quote:UPDATE 2: Also, PERL and especially CPAN are absolutely terrible... Turn away from the dark side ... Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.23 18:57:00 -
[231] - Quote
I am not getting any errors, but every so often I will open the uploaded and it will not upload anything (0.4). Restarting does not help and I do not have any other uploaders running. It was working last night before downtime, but today it is not uploading anything.
PS: I am disappointed that you didn't make a version 3D. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
42
|
Posted - 2011.12.23 19:01:00 -
[232] - Quote
There simply might not be anything to upload (even if the upload suggestions are still happily cycling, if nothing has changed since last time, nothing will be uploaded). The EVE client won't even write a cache file, so the uploader does nothing at all. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.23 19:22:00 -
[233] - Quote
Callean Drevus wrote:There simply might not be anything to upload (even if the upload suggestions are still happily cycling, if nothing has changed since last time, nothing will be uploaded). The EVE client won't even write a cache file, so the uploader does nothing at all. That was it, good to know. I cleared my cache just to see if that was it, and it was. I thought the uploader deleted the cache files after uploading them. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
42
|
Posted - 2011.12.23 19:35:00 -
[234] - Quote
Controller Vrelk wrote:That was it, good to know. I cleared my cache just to see if that was it, and it was. I thought the uploader deleted the cache files after uploading them.
It does delete the actual file, but it doesn't clear the same cached information inside the EVE client, which for some reason also saves a copy of the data in it's memory
Why it writes the cache file to disk and additionally stores an in memory copy in addition is a big questionmark to me, but I'll happily make use of it Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.23 21:36:00 -
[235] - Quote
Hmm. I left and had the market scanner running for about 2-3 hours. I just got home and found that the uploader had only uploaded 20 things, and this is after I cleared my cache.
EDIT: The market scanner on you site seems to be going in circles, it is scanning the same things over and over. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
42
|
Posted - 2011.12.23 21:48:00 -
[236] - Quote
Are you certain it is going in circles? I cannot duplicate this behaviour... It should be easy to test by seeing if the items that are shown first are different if you refresh the page. Developer/Creator of EVE Marketeer
|
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.23 21:52:00 -
[237] - Quote
Callean Drevus wrote:Are you certain it is going in circles? I cannot duplicate this behaviour... It should be easy to test by seeing if the items that are shown first are different if you refresh the page.
When I left I saw a weathe blueprint and a quantum flux generator 3 go by, when I came back and noticed that it had uploaded 20 items I watched for a minute and saw the same things go by. |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
2
|
Posted - 2011.12.24 07:35:00 -
[238] - Quote
Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed
4200 uploads to EMD and EC, 407 to EMK |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
42
|
Posted - 2011.12.24 09:52:00 -
[239] - Quote
Hmm, for some reason mysql decided that it'd had enough. More, the server seems to have decided it was enough, and it has been rebooted by my hosting provider because it was unresponsive. (Now, that is awesome, except that mysql didn't work afterwards). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
461
|
Posted - 2011.12.24 13:16:00 -
[240] - Quote
Windows XP error (trying to install on the laptop, where it's never been installed before).
Traceback (most recent call last): File "emkUploader.py", line 19, in (module) File "wx\_core.pyo", line 7981, in __init__ File "wx\_core.pyo", line 7555, in _BootstrapApp File "emkUploader.py", line 11, in OnInit File "UploaderFrame.pyo", line 96, in __init__ File "UploaderFrame.pyo", line 150, in initialize File "UploaderFrame.pyo", line 344, in installationAutodetectButtonClick File "os.pyo", line 423, in __getitem__ KeyError: 'LOCALAPPDATA' |
|
Scrapyard Bob
EVE University Ivy League
462
|
Posted - 2011.12.24 13:54:00 -
[241] - Quote
I figured out a new way to break the API call by putting garbage in for the type IDs.
http://www.evemarketeer.com/api/info/32_#N/A_33_#N/A_#N/A_#N/A_#N/A_#N/A_#N/A_#N/A/xml/10000002/sell_lowest5
This results in the error:
MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ') AND station_id = 0 ORDER BY FIELD(type_id, 32,)' at line 1
Which may possibly mean that you are vulnerable to a SQL injection. You'll need to ensure that those values are forced into numeric types before passing them to the SQL engine. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
43
|
Posted - 2011.12.25 08:56:00 -
[242] - Quote
They are escaped before adding them into the query, that said, it's a bit weird they actually break it, but probably because the query expects them all to be numbers.
Thad said, I"ve added it to the feedback page and will see if I can fix it.
As for the Win XP error. I guess that's what I get for trying to lend stuff from EMD, the odd part is that it's actually designed to do that differently for XP, so I guessed it should work. Will fix in next version. Developer/Creator of EVE Marketeer
|
Invictra Atreides
Toward the Terra
16
|
Posted - 2011.12.27 01:44:00 -
[243] - Quote
Today I just started to use your uploader and the XML API. Now when I run the uploader it does its thing and uploads the files, but when I check on evemarketeer.com the data is not updated. It still shows the old data. why is that ?
It also uploads to eve-central.com so I also checked there and the data there was updated.
Then I also checked eve-marketdata.com and the data there was also updated, but their XML API was still showing the old data.
Is this a normal behavior? I don't understand. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
43
|
Posted - 2011.12.27 01:52:00 -
[244] - Quote
Invictra Atreides wrote:Is this a normal behavior? I don't understand.
It both is, and is not at the same time. The server was rebooted yesterday, and wasn't running any processing threads due to them all being killed because of that. It should be updating normally now. Anyhow, I really need to start doing this automatically, as these are situations I like to avoid, so hopefully this problem won't occur again.
If you ever wonder whether anything is happening to your updates, you can look at the upload statistics page. If the EVE Marketeer backlog counter keeps increasing, something might be wrong with the processing (but your uploads are still saved, and will be processed when the problem is resolved). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
463
|
Posted - 2011.12.27 14:49:00 -
[245] - Quote
http://www.evemarketeer.com/api/info/34_35_36_37_38_39_40_11399/xml/10000002/sell_lowest5
Is showing zero data for Pyerite, Mexallon and Zydrine this morning.
http://www.evemarketeer.com/item/info/pyerite http://www.evemarketeer.com/item/info/zydrine http://www.evemarketeer.com/item/info/mexallon
|
Invictra Atreides
Toward the Terra
16
|
Posted - 2011.12.27 21:24:00 -
[246] - Quote
It seems half the database was corrupted. The restore data on the webpage helps.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
44
|
Posted - 2011.12.28 10:13:00 -
[247] - Quote
Well, it isn't that the database is corrupted, it is more that EVE Central keeps sending empty data at times. I thought I'd fixed the problem where empty EVE Central data was sent, but it seems not. I built in the restore button in any case, so you can go back to correct data if you think the data can't possibly be right.
I'll see what is happening to make some data still being cleared. Developer/Creator of EVE Marketeer
|
TheSmokingHertog
Black Hole INC
3
|
Posted - 2011.12.28 11:16:00 -
[248] - Quote
Just found out about your site / tools. Thx for all efforts. Lets see how to put the information to good use over here :). |
Kaedaar OLeary
ADK Industries
0
|
Posted - 2011.12.29 01:48:00 -
[249] - Quote
Any idea how I can upload to the battleclinic servers as well. I see you can add more endpoints in your uploader.
Thanks for all your efforts.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
47
|
Posted - 2011.12.29 09:36:00 -
[250] - Quote
I am not aware of battleclinic supporting direct uploading. If and when they implement one of the upload formats, they will publish an endpoint url that you can add to the uploader. Developer/Creator of EVE Marketeer
|
|
Rektide
School of Applied Knowledge Caldari State
0
|
Posted - 2011.12.29 17:54:00 -
[251] - Quote
Hi. I'd like to request some kind of streaming output. EVE Metrics had an AMQP feed, EVE Central has an email list that sends out an email with all of the raw CSV received in the last minute: I'd love something with that push-based streaming out in EVE Marketteer.
There is recent_updates, but it's something that needs to be polled & updated manually. I'm also afraid of missing contents, so I'd have to request time_since_last+1 then dedupe, which adds a little cost & complexity.
Connectivity options: email, amqp, webhook callbacks, xmpp MUC bot, zeromq, ... |
Dragonaire
Corax. PURgE Alliance
25
|
Posted - 2011.12.30 04:43:00 -
[252] - Quote
Might check out this thread as there are plans to have a unified exchange format that is being developed that all the upload sites and others are working on implementing including EMK https://forums.eveonline.com/default.aspx?g=posts&t=18466&find=unread Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Scrapyard Bob
EVE University Ivy League
570
|
Posted - 2012.01.01 01:51:00 -
[253] - Quote
On the market scanner page where it shows "(6857 / 6922 updated in last 24 hours)" - that number does not appear to be updating as time goes on. So, for instance if it shows 12% has been updated in the last 24 hours, then you look at the regional page it will say something far higher after a few hours.
Also, the regional status page does not always render the first time visited in Firefox. I have to go to that URL, then force a page reload in order for it to display properly. It's a hit/miss error, possibly cache related. It may only happen once per hour (as the page gets updated).
http://www.evemarketeer.com/statistics/regions
PS: The unified uploader is running happily, 174k cache files processed since it started and no crashes. There does seem to be a memory leak somewhere because it's up to 257MB of "peak working memory" with 183MB in current use (174MB private/commit size). It started at around 35-40MB when first launched. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
63
|
Posted - 2012.01.01 18:11:00 -
[254] - Quote
@Rektide: There are still plans for streaming output using AMQP, but at the moment it's not very high on my priority list. I might be able to simulate something EVE Central like for the interval though.
@Scrapyard: I know, there is something going wrong with the way those values are calculated. I'll investigate the issue. Same goes for regions, though I have no clue as to what is causing that yet.
Good to hear that the unified uploader is running happily. I wonder what is causing the large memory consumption though. How long have you had it running? Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
575
|
Posted - 2012.01.01 23:12:00 -
[255] - Quote
Callean Drevus wrote: Good to hear that the unified uploader is running happily. I wonder what is causing the large memory consumption though. How long have you had it running?
A a guess, somewhere around 9 days and 174,000 cache files processed. So it's a fairly slow memory leak. I have 16GB of RAM, but I could see that being an issue for others.
The ".log" files don't put time-stamps in when they start-up (it just says "Logger started").
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
63
|
Posted - 2012.01.03 23:17:00 -
[256] - Quote
Ok, I guess that means it's not a very radical problem, and it explains why it hasn't shown up in my tests. Will still see whether I can do something about that, but my guess is most people don't leave their computers on for 9 days ;)
In addition, if any of you ever wondered what your wallet balance graph looks like, which I guess you didn't since we all know of EVE Mentat and all those programs, I've always felt that such a function was missing on EVE Marketeer, so I've added it.
Find it under My Market -> Wallet History Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
581
|
Posted - 2012.01.04 08:13:00 -
[257] - Quote
And a small bump because the history tab is showing chart history that only goes through Nov 27th.
http://www.evemarketeer.com/item/info/pyerite
(It's probably already on your to-do list.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.04 10:38:00 -
[258] - Quote
And you did upload more history than November I take it? :P It wasn't on my to do list yet thanks for letting me know. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
582
|
Posted - 2012.01.04 14:18:00 -
[259] - Quote
Which reminds me of a question. For eve-marketdata, you had to manually switch your market screen to be looking at the "history" tab in order for it to gather data before running scans. Is that still true that the market history cache files will not be generated if you do not specifically switch to the "history" tab?
|
Kaladr
Dreddit Test Alliance Please Ignore
13
|
Posted - 2012.01.05 00:32:00 -
[260] - Quote
Yes, the history tab needs to be open for the client to make calls to get history (and in turn end up in the cache files).
EVEC still has no history interface, so of course no uploads go to it. Maybe soon, but there is a lot of core stuff in the primary "orders" land which is still taking up my time. Creator of EVE-Central.com, the longest running EVE Market Aggregator |
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.01.05 03:57:00 -
[261] - Quote
When you get a history with a single entry, and 0 orders/0 quantity, can you somehow depriorotise that item for that region, so it is hardly ever requested when scanning the market?
Currently scanning Branch, and it feels like 90% of the time I'm scanning for items that do not exist in the region, and never have :P |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.01.05 04:37:00 -
[262] - Quote
Oh, and when uploading item history, nothing seems to be coming up in my upload history? |
Maeste Madeveda
The Spawning Pool Team Liquid
1
|
Posted - 2012.01.05 15:38:00 -
[263] - Quote
Would it be possible to add the 'field_filter' component to the 'station_info' (station details) request ?
I'd like to get a particular have info as precise as possible and sometimes artificial orders are messing up with the values of the item I want in the simpler item details.
Also it is possible to have the option not to delete the files updated through the unified uploader like it was in the previous uploader ? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.05 18:58:00 -
[264] - Quote
Quote:Yes, the history tab needs to be open for the client to make calls to get history (and in turn end up in the cache files). Thanks for answering that question :)
Quote:When you get a history with a single entry, and 0 orders/0 quantity, can you somehow depriorotise that item for that region, so it is hardly ever requested when scanning the market? Yes, this seems like a smart idea. Will probably try something along the lines of what Scrapyard Bob described a while back.
Quote:Oh, and when uploading item history, nothing seems to be coming up in my upload history? If you are using uploader 0.4, you should switch to 0.4b, but I assume you have done that already. I'll investigate this issue.
Quote:Would it be possible to add the 'field_filter' component to the 'station_info' (station details) request ?
I'd like to get a particular have info as precise as possible and sometimes artificial orders are messing up with the values of the item I want in the simpler item details. Yes, I'll add the field filter, I was under the impression it was there already, but it's not. Can you describe artificial order? I do my best to remove all artificial orders, but maybe someone came up with an idea good enough to pass through the filters .
It's also possible to not delete the files, but it'll probably move them to a different folder then, since it's going to interfere with the uploading otherwise (especially if people leave their client running for weeks on end , it's bound to get slower). Developer/Creator of EVE Marketeer
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.01.06 22:25:00 -
[265] - Quote
yep, 0.4b. Well 99% sure. I download the file called 0.4b from the site, and unpacked it. While it's running, however, the unified uploader says "Current Version: 0.4"
Oh also, a small thing with the unified uploader, the text where it shows the uploads, everage time, etc for ever marketeer, eve central and even marketdata...... that text is *constantly* flashing. Sorta gives me a headache if I look at it for too long. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.06 22:29:00 -
[266] - Quote
Yeah, it's version 0.4b. Probably forgot to change the version numbering (again).
The text is constantly flashing because it is being updated... I can lower the frequency though, which should make it better. On the other hand, I can hardly believe you actually spend time staring at the loveliness of uploading Developer/Creator of EVE Marketeer
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.01.07 02:29:00 -
[267] - Quote
it's.....so.... mesmerizing......
actually, I'm just glad it doesn't freeze up like the old uploader :P |
Scrapyard Bob
EVE University Ivy League
586
|
Posted - 2012.01.07 03:57:00 -
[268] - Quote
One request that came in through a private convo today with someone. This particular user also runs their own cache-scanning software.
The new uploader no longer has the option of "leave cache files in place" (i.e. "don't delete"). Or maybe we need an option of "delete after N hours".
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.07 09:25:00 -
[269] - Quote
Yup, I already got that 4 posts ago it will be added. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
586
|
Posted - 2012.01.07 15:09:00 -
[270] - Quote
Ooops, guess I shouldn't post requests when running on 3 hours of sleep.
On the history graph, the little tool-tip that shows you the price value as you mouse-over the lines on the graph is very hard to read. Can you make the background more opaque and increase the contrast of the text color in relationship to the background? |
|
Scrapyard Bob
EVE University Ivy League
586
|
Posted - 2012.01.09 00:09:00 -
[271] - Quote
(big list incoming)
Also, on the history graph, it needs to filter out outliers before setting the scale. A good example is the orca price in the forge.
http://www.evemarketeer.com/item/info/orca - Someone put one up for sale in Jita at about 10x above the other prices, pushing the scale out of whack
...
Group summary links are still non-functional. Which kind of goes back to my previous suggestion of showing the item name along with the 5% buy sim price for the top (4) market regions on each row. So when I look at:
http://www.evemarketeer.com/item/group_summary/18
I would see:
(item) The Forge / Domain / Sinq Laison / Heimatar Tritanium - 4.35 / 3.94 / 3.38 / 3.90 Pyerite - 5.33 / 5.30 / 4.96 / 5.11 etc.
If the average of values are above 100k, then it should switch to displaying the entire page in thousands, above 100M, in millions.
For the parent group (which does not contain actual items, only other market groups) it wouldn't make sense to display the prices. For instance, the parent group of "Materials" or "Ore & Minerals".
http://www.evemarketeer.com/item/group_summary/1031 http://www.evemarketeer.com/item/group_summary/533
...
On the regional drop-down, it would be nice if you would add "meta" regions like:
-1 = All -2 = High-Sec Only -3 = Lo-Sec Only -4 = High/Low Only
(Swiped from EVE-MarketData's URLs for prices.) |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.01.10 11:08:00 -
[272] - Quote
not sure if you've seen this site http://www.eve-market-guide.com
it's a great service they offer, but the data they use is never up to date. If possible, this service would make a great addition to the member services on offer. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.10 17:32:00 -
[273] - Quote
Hadn't seen eve market guide yet. Will loot their functions :P though EMK has everything but their invention prices, it's not yet as accessible.
As to scrapyard, good suggestions, will do. Sorry, haven't been able to work on EMK much the past week Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.10 21:41:00 -
[274] - Quote
Ok:
- Field filter for station API - Fixed orca price graph, and hopefully other outliers that are too mad to be true - Added the group summary pages, will still need some (lots of) work - History mouse over is more readable
These were the things that were easy to add :) Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
598
|
Posted - 2012.01.11 00:32:00 -
[275] - Quote
Group summary pages look very good, other then its choice of the (4) regions to show for minerals:
http://www.evemarketeer.com/item/group_summary/18 - The Forge (yep, there's Jita) - Delve (uh...) - Heimatar (yep, that's Rens) - Domain (yep, Amarr)
http://www.evemarketeer.com/item/group_summary/476 - That one uses the (4) recognized market hub regions.
I'm thinking you may want to just force the top (4) regions. Or limit your heuristic to empire regions with data less then 15 days old (Delve is 45-60 days out of date).
|
Scrapyard Bob
EVE University Ivy League
598
|
Posted - 2012.01.11 00:44:00 -
[276] - Quote
One more thing (hah, right) for the group summary page - which is very nice :)
The ability to click on one of the numbers to take me to that region's sell page so I can see the sell orders for that item in that region.
Okay, make that two things.
It would be useful if the page title would change depending on what you're viewing. So when looking at:
"EVE Marketeer - Group Summary - Standard Battlecruisers - Minmatar" http://www.evemarketeer.com/item/group_summary/473
"EVE Marketeer - Item - Tornado" http://www.evemarketeer.com/item/info/Tornado |
Scrapyard Bob
EVE University Ivy League
598
|
Posted - 2012.01.11 05:15:00 -
[277] - Quote
If you're interested in seeing market history uploaded more often, the regional status page needs to report on this.
- Drop the "% with last year" column. If the data hasn't been updated within the past month, it's not very useful at all. Have the Last hour/day/week/month fields only report on buy/sell order data freshness. Drop any items which have not had updates within the previous 365 days from the average as they're items that you'll probably never see.
- Add a new column for how fresh regional history data is and put it after the "Avg Time Ago" number. Call it "History Age" or "Order History". Again, drop anything older then 365 days before calculating the average age.
- The market scanner page will need to report both how fresh the buy/sell order data is (average age) as well as displaying the average history data age. I suggest always displaying both numbers rather then doing it like EMD where you have to flip to the other page to see how fresh the history data is. If both numbers are "in my face" when I hit the market scanner page, I can make an educated decision whether to set the market window to history or order details for this particular session. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.11 09:26:00 -
[278] - Quote
Lolz, yeah, there was something wrong with the algorithm that is used to determine the top regions. I just couldn't get it right, so I'll probably allow people to set their own regions there, it's just going to drive me mad otherwise. The default would then be to show the top 4 general market regions.
I didn't do sell/buy averaging yet because sometimes either of the values is zero, and that would mess up the average. Catching that will take a little more time, but don't worry, will be done.
Changing page titles, smart idea! Currently they only change based on the page you're on, but it's perfectly possible to add additional information.
Going to the right sell region automatically is more difficult, though I guess I can solve that in some way (it's not that is actually difficult, it's just that I"ve made it so :P).
Market history needs some love indeed. The reason I didn't do this yet is that a while ago, CCP released a dump of their history data, and was talking about providing all of that automatically, which I thought was pretty awesome, as it would erase the need for history uploads. Haven't heard anything about that anymore though.
Developer/Creator of EVE Marketeer
|
TheSmokingHertog
Black Hole INC
12
|
Posted - 2012.01.11 16:11:00 -
[279] - Quote
Scrapyard Bob wrote:Group summary pages look very good, other then its choice of the (4) regions to show for minerals: http://www.evemarketeer.com/item/group_summary/18- The Forge (yep, there's Jita) - Delve (uh...) - Heimatar (yep, that's Rens) - Domain (yep, Amarr) ..
How to set another region in this overview? Like delve?
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
64
|
Posted - 2012.01.11 16:20:00 -
[280] - Quote
The version that is currently online determines the regions based on market activity (or it should, anyway), but I've developed a new version today that allows you to set your own regions. I'll put that version online shortly, first I'll finish my awesome new table engine. Developer/Creator of EVE Marketeer
|
|
Scrapyard Bob
EVE University Ivy League
602
|
Posted - 2012.01.11 23:03:00 -
[281] - Quote
Nice changes to the group summary pages (including links and customizable region selection).
I think it's a toss-up whether the group summary pages should be sorted by name or TypeID. For the Minerals page, TypeID sorting makes more sense. But for a page like the following, sorting by name would make more sense:
http://www.evemarketeer.com/item/group_summary/265
The gripping hand options might be:
- TypeID sort in cases where there are less then 10 items in the group. - Or a custom list of groups where they should be sorted by TypeID. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
67
|
Posted - 2012.01.11 23:22:00 -
[282] - Quote
Uh... the forums ate my previous post announcing the latest changes...
In any case, sorting by name is probably the way to go. People will start searching for a name as soon as they find that almost everything is sorted by name, and then a few groups with custom typeID sorting will probably not make much sense.
In any case, this page will be ported to the new table grid as well, which allows sorting of any kind. Just have to add a column for typeID. Developer/Creator of EVE Marketeer
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.01.12 01:50:00 -
[283] - Quote
Callean Drevus wrote:Will loot their functions :P
Another site, if you're intereted
https://forums.eveonline.com/default.aspx?g=posts&t=28048&find=unread
EVEWalletAware, apparently was much loved. If you're looking for more functions to loot, might be worth a looksie. Also, since that program has stopped it's support, you might get more happy users for evemarketeer |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
68
|
Posted - 2012.01.12 12:06:00 -
[284] - Quote
Yeah I've long been planning to grab every fuction I could find in any market program anywhere. It's just that these other things keep comin up. My attention seems to swing to whatever is closest at the moment though.
A while back someone asked for something to calculate character value, and yesterday I suddenly felt like I could no longer do nothing with that request, simply because I was curious what my own value would be . The page looks butt ugly though (My Characters). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
605
|
Posted - 2012.01.13 03:39:00 -
[285] - Quote
On the group summary pages; right now you have "buy" and "sell" in the drop down menu. Can this be expanded to:
http://www.evemarketeer.com/item/group_summary/1337/sell
"Sell (5% simulated buy)" - the default (or remember it if I have a login profile) "Sell (lowest)" "30-day average" "Buy (5% simulated sell)" "Buy (highest)" - the current default, which I always change to "sell (lowest)"
Which leads in to a question about the "item API". Now that you're starting to gather information on history, it might be useful to calculate for each item a 30-day average taking the order history within the past 30 days and giving us:
- average # units sold per day - average price per item sold
i.e.
http://www.evemarketeer.com/api/info/34/xml/10000043/30d_units_per_day http://www.evemarketeer.com/api/info/34/xml/10000043/30d_avg_price
That would give us an idea of how many units are changing hands per day over the past month in each region. |
Trin Dainos
0
|
Posted - 2012.01.13 13:13:00 -
[286] - Quote
Is there a problem with "Best Items"? It seems a litle bit empty..... |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
68
|
Posted - 2012.01.13 16:54:00 -
[287] - Quote
Can do! Was already planning to add the featur that remembers your settings, for all those kind of useful dropdowns that currently keep forgetting your settings :)
I've also been calculating that information (averages) for a while, it was just never exposed. I'll not include it in the info API though, since I'm never sure about history information, and always sure about what is currently in the Info API. Will do what I can about adding it though. Probably first a separate API.
Best items is sometimes a bit fickle, especially when a day has just begun. The interval shouldn't be too long though... Developer/Creator of EVE Marketeer
|
Trin Dainos
0
|
Posted - 2012.01.13 17:05:00 -
[288] - Quote
Callean Drevus wrote:Can do! Was already planning to add the featur that remembers your settings, for all those kind of useful dropdowns that currently keep forgetting your settings :)
I've also been calculating that information (averages) for a while, it was just never exposed. I'll not include it in the info API though, since I'm never sure about history information, and always sure about what is currently in the Info API. Will do what I can about adding it though. Probably first a separate API.
Best items is sometimes a bit fickle, especially when a day has just begun. The interval shouldn't be too long though...
The list is empty for me since nearly 24h. In everey region....
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
68
|
Posted - 2012.01.13 18:58:00 -
[289] - Quote
It might have had something to do with a completely different problem that the website was suffering from. Which is now fixed. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
68
|
Posted - 2012.01.14 11:27:00 -
[290] - Quote
Top Uploads
EVE Central 581214 Scrapyard Bob284262 oshua 133149
Those two have been doing most of the uploads in the entirety of the EVE universe for the past week. The EVE Central number is all the orders I retrieved from EVE Central, whether they were also uploaded to EMK or not.
This means that there (if these were both uploading to both sites) were only 163803 uploads that were uploaded uniquely to EVE Central. You've got to love multiple uploading ^^
Much respect to the people whom are doing this great service to the community by providing half of ALL the order data the market sites receive by themselves. Developer/Creator of EVE Marketeer
|
|
Maeste Madeveda
The Spawning Pool Team Liquid
1
|
Posted - 2012.01.14 17:43:00 -
[291] - Quote
Both of them have multiple accounts who are running the scanner all day long. I only have one account so I can't have the same numbers. But during the last month, I managed to get to the 13th place of all time uploaders mostly by uploading in Heimatar. I believe everyone can have a influence particularly in they trade outside of the 4 main racial hubs who are updated regularly. It would be very good to have inputs on every Empire region. That should be our goal actually.
I like the new changes, some of them might need tweaking but I need to use them more before giving an appropriate feedback. |
Scrapyard Bob
EVE University Ivy League
608
|
Posted - 2012.01.14 19:38:00 -
[292] - Quote
Depends on the day. I always try and make a point to update (3) regions at night (Heim/Metro/Sinq), but I'll sometimes run the scanner during the day for Heimatar/Sinq. Especially if the market is chaotic that week with lots of price swings through the day. I'll also sometimes float the 3rd alt's account around and hit up different regions.
We definitely need more people in places like Everyshore, Tash-Murkon, Domain, Citadel, Lonetrek, etc
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
68
|
Posted - 2012.01.15 09:31:00 -
[293] - Quote
Quote:Both of them have multiple accounts who are running the scanner all day long. I only have one account so I can't have the same numbers. I didn't mean to say other people didn't do their best I just found it incredibly funny that two people are actually doing half of all the uploading Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
543
|
Posted - 2012.01.15 11:34:00 -
[294] - Quote
This site just keeps getting better and better! Nice work! |
TheSmokingHertog
Black Hole INC
13
|
Posted - 2012.01.15 16:01:00 -
[295] - Quote
Thx for all the grt efforts, the summeries of sales are great. |
Scrapyard Bob
EVE University Ivy League
609
|
Posted - 2012.01.15 17:52:00 -
[296] - Quote
Also, maybe the "top uploaders" list should be done based off of credits rather then raw uploads.
Then you could award more/less credits in order to skew towards the behavior that you want. (Looking at the upload history, this already seems to happen somewhat. Sometimes I get 0.1 credits, sometimes I get 5-15 credits per upload item.)
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
69
|
Posted - 2012.01.15 19:57:00 -
[297] - Quote
The red green lines mean whether your orders are the lowest/highest in the region/station they are sold/bought in, depending on whether you selected station range, and whether they are buy or sell orders.
@Scrapyard: Can you possibly have missed the fact that the second table on the top uploaders page is sorted by amount of credits? The credit system works as you describe , the minimum value is 0.1, for an upload which add absolutely nothing (see, repeat uploading), to 25 for an upload that redefines the stats for the entire region. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
609
|
Posted - 2012.01.15 20:51:00 -
[298] - Quote
Callean Drevus wrote:@Scrapyard: Can you possibly have missed the fact that the second table on the top uploaders page is sorted by amount of credits? The credit system works as you describe , the minimum value is 0.1, for an upload which add absolutely nothing (see, repeat uploading), to 25 for an upload that redefines the stats for the entire region.
Yes, but does that control who shows up on the front page? Or is that off of raw upload count?
(And thanks, I was curious how some of the scores were calculated.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
69
|
Posted - 2012.01.16 06:55:00 -
[299] - Quote
Ah, true. I'm not entirely certain which one I use for the front page, but I assume it's upload count. Mostly because showing an arbitrary number of credits doesn't mean anything to the uninitiated. Maybe it can be changed to 'best uploaders', though not entirely certain how to make that comprehensible. Developer/Creator of EVE Marketeer
|
Kaladr
Dreddit Test Alliance Please Ignore
18
|
Posted - 2012.01.16 07:06:00 -
[300] - Quote
Unified Uploader bug report (or so I think, can't tell if this is the unified upload or not):
Looks like a POST request is being made to the resource "http://eve-central.com/...", as below:
xx - - [15/Jan/2012:10:16:34 +0000] "POST http://eve-central.com/datainput.py/inputdata HTTP/1.1" 200 120 "-" "-"
However, the http:// doesn't belong in the actual request portion as below:
xx- - [15/Jan/2012:10:16:34 +0000] "POST /datainput.py/inputdata HTTP/1.1" 200 120 "-" "-"
Luckily enough server bits reformat the request into something sensible, but I can't guarantee this behavior when I switch over to the "new stack". Could you check the unified uploader for me to make sure the POST URL is sensible? Creator of EVE-Central.com, the longest running EVE Market Aggregator |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
70
|
Posted - 2012.01.16 07:12:00 -
[301] - Quote
It seems sensible enough, just wrong. But I'll take a look at the unified uploader, and fix this. Thanks for the report :) Developer/Creator of EVE Marketeer
|
6ie
State War Academy Caldari State
0
|
Posted - 2012.01.16 16:58:00 -
[302] - Quote
I run 3 eve clients at once, all scanning the market. The cache folder grows to 5,000 files, then EVE starts deleting them. I think the uploader runs ok when I have 1 client running. Possibly the uploader cant handle 3 clients at once?
Zeropoint reader shows about 99% of the files are being deleted, but either they are not, or they are being missed by the uploader. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
70
|
Posted - 2012.01.16 17:19:00 -
[303] - Quote
Can you check the log files in the installation folder and see if you find anything unusual? Developer/Creator of EVE Marketeer
|
6ie
State War Academy Caldari State
0
|
Posted - 2012.01.16 17:49:00 -
[304] - Quote
(Using: Unified 0.4b) This is the folder that has the 5000 files: MachoNet\87.237.38.200\303\CachedObjects
All 4 log files in the uploader dir just read: Logger started
|
Scrapyard Bob
EVE University Ivy League
609
|
Posted - 2012.01.16 19:31:00 -
[305] - Quote
6ie wrote:Possibly the uploader cant handle 3 clients at once?
It shouldn't have an issue like that - I used to break 0.3 and 0.4 all the time with running (3) clients. But since 0.4b, haven't had a lick of trouble with the uploader (even with 3 clients, all feeding the same cache folder, or running off of diff cache folders, same region, diff region). So far, 5 days running, 235k files read, 234k files deleted by the Zeropoint reader.
But I'm on Win7 64bit.
I show about 20.4k files in my MachoNet cached objects folder. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
70
|
Posted - 2012.01.16 20:20:00 -
[306] - Quote
Ah, yes, the uploader doesn't really care about CachedObjects at all, the only thing it's interested in is CachedMethodCalls, which should be empty.
I take it it is uploading okay otherwise? With the exception of your CachedObjects folder staying full (which the uploader is not having any effect on)?
Also EVE Central might have some trouble keeping up with the uploads you are doing if you're running 3 clients at the same time, it'll just permanently stay at a queue of 50 items, and discard any excess.
@Kaladr: Fixed the URL scheme, will be included in the next iteration of the UU.
UPDATE: For those of you reading this, EMK likes it when you request item information in batches of 25 instead of one at a time :P or even better, if you're going to request them anyway, all the items in the region at the same time (and discard some of the information).
Also, I wonder who the person is that is uploading 4 orders per second with (seemingly) the old uploader, may have to add the user agent string to the uploader too. Would be able to make more sense of the access log that way. Developer/Creator of EVE Marketeer
|
6ie
State War Academy Caldari State
0
|
Posted - 2012.01.16 21:48:00 -
[307] - Quote
Ah ok, yeah i see the files come and go from CachedMethodCalls. I also see EVE Central: unable to upload in the log I don't know why CachedObjects gets so many objects - I guess I will just del them periodically.
Is there a way you can code the uploader can not del the cache files for X minutes? I like to import the cache files to Evementat (and eventually to a MYSQL DB with Reverence), but the cache files get deleted by the uploader. (or add an option to no del them at all). |
Kaladr
Dreddit Test Alliance Please Ignore
19
|
Posted - 2012.01.16 22:36:00 -
[308] - Quote
Callean Drevus wrote: Also EVE Central might have some trouble keeping up with the uploads you are doing if you're running 3 clients at the same time, it'll just permanently stay at a queue of 50 items, and discard any excess.
@Kaladr: Fixed the URL scheme, will be included in the next iteration of the UU.
Thanks :)
And I demand of myself to fix the upload performance hole.
Creator of EVE-Central.com, the longest running EVE Market Aggregator |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
71
|
Posted - 2012.01.17 06:54:00 -
[309] - Quote
Jup, the next thing I will add to the uploader is the ability to either move or not delete the cache files. It was just the easiest solution to delete them at first :) Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
550
|
Posted - 2012.01.17 08:45:00 -
[310] - Quote
Please forgive me if you have already answered this but...
Is there a way by chance to develop something in EM that helps you find the best use of LP? I realize that it will mean manually inputting prices for the faction items but I was curious if it is possible to do it with this? |
|
6ie
State War Academy Caldari State
0
|
Posted - 2012.01.17 15:41:00 -
[311] - Quote
Callean Drevus wrote:Jup, the next thing I will add to the uploader is the ability to either move or not delete the cache files. It was just the easiest solution to delete them at first :)
Nice, thanks :) BTW, how are the credits per upload calculated? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
71
|
Posted - 2012.01.17 20:09:00 -
[312] - Quote
Damn, the forums ate my post again :S
As far as I know there is no way to retrieve either LP or faction prices (contracts) from the database. That means I could build something that allows you to input your own data, but it'd be kind of pointless. If the data is somewhere in the SSD though, it could be useful.
@6ie: Is mentioned somewhere on the previous page, basically means, the more influence your uploads have on the state of the market, the higher the reward. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
615
|
Posted - 2012.01.18 04:02:00 -
[313] - Quote
http://prices.c0rporation.com/
Looks like you have to hand-submit prices for items sold via contracts. Not sure whether viewing a contract causes it to be written out to a cache file.
(Can't think of any other faction price sites off-hand.) |
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
562
|
Posted - 2012.01.18 06:50:00 -
[314] - Quote
Callean Drevus wrote:Damn, the forums ate my post again :S
As far as I know there is no way to retrieve either LP or faction prices (contracts) from the database. That means I could build something that allows you to input your own data, but it'd be kind of pointless. If the data is somewhere in the SSD though, it could be useful.
@6ie: Is mentioned somewhere on the previous page, basically means, the more influence your uploads have on the state of the market, the higher the reward.
I would figure an input window that says "Enter sell price" For a group of LP store items of your choosing. It then knows what said items cost in LP isk and then goes to the market DB to find region prices for tags and other items that can be retrieved.
Then it will tell you LP efficiency from those group of prices.
I know it wont be very accurate. Just faster than having a spreadsheet or having to write down the tag prices every time I want to do a rough calculation. |
Trin Dainos
0
|
Posted - 2012.01.18 11:03:00 -
[315] - Quote
Don-¦t know if this question has been asked already, but I-¦d like to know how you get the movement data? Do you just use the historical date you got? But than also canceled orders would be in there, right? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
72
|
Posted - 2012.01.18 11:23:00 -
[316] - Quote
The history data that is provided by CCP is based on transactions, so that is a fairly accurate indicator of the movement of the market. I do some more interesting things, like checking whether the average over time is closer to the highest or lowest price to determine whether there are more buy or sell orders being filled.
Today I found out that Jitanomic, the bot driven market information website had the information on loyalty points. I've tried to find it in other places, but wasn't able to find it, which must mean that it isn't normally available. Having no compunctions whatsoever about leeching off a known botter, I proceeded to scrape the website, which is now happily running :P
Hopefully means I will have some information on loyalty points, making only the faction item prices an issue of manual input. Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
564
|
Posted - 2012.01.18 11:33:00 -
[317] - Quote
Um instead of getting that info from a site you say is run from bots why not ask CCP if there is a way to gather that info from API or something?
And I don't mind putting in the amounts info a box during the calculation if that is needed.
Edit: Seriously how about we just send you the data on the LP items and you can update it. 90 percent of the battle is diving into the market and running the calculations to find isk per lp. Anyone actually using the data should check to make sure the requirements are the same before buying the LP item.
That would allow you to avoid having to use another site's data. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
72
|
Posted - 2012.01.18 11:46:00 -
[318] - Quote
The reason for that is pretty simple. I've found no way to get the data from the API, nor from the static data dump. CCP doesn't make a habit of providing data to a single person (as in I've never heard of it happening), so the easiest solution available is getting the data from someone else.
I could ask CCP whether I have missed something, but I estimate they'll answer with 'not possible'.
This saves me both the time and trouble looking for it. That said, will have to verify it is actually accurate, there is a number of websites containing the information already, but they're either not up to date, or inaccurate (entered by players). Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
564
|
Posted - 2012.01.18 12:13:00 -
[319] - Quote
How about you can start the feature with having us input the stuff ourselves while you make a topic in the forum asking if said data is retrievable?
Then maybe you can see if people will send screenshots of the LP page and use that to start filling in a database. If someone wants something added they need to provide a screenshot of the LP store showing the item in question. Tho I will be happy to move around to help fill holes if it is requested.
I personally think it is a hell of alot better than relying on any kind of bot to do this. And community involvement will help grow your site in my opinion.
If it falls flat on its face you could always go back to your original plan right?
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
564
|
Posted - 2012.01.18 12:26:00 -
[320] - Quote
Wanted to show an example of what I was talking about
http://i.imgur.com/tz5ZV.jpg That was done in about 30 seconds copy pasta into paint with a crop. What I can do in gimp is put one on top of the other to get a big image that spans half or all the LP store at once. Then use extreme JPEG compression to keep it small. |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
74
|
Posted - 2012.01.18 19:50:00 -
[321] - Quote
You must be kidding, I'm not going to do that to myself (creating OCR?), or to you guys.
I should probably make something clear. I'm not relying on ANY bot to get the data.
I've retrieved it once from the website, and now I can use it forever (unless CCP changes LP prices). In addition, for all I know the original data was obtained in a perfectly legal way, there is no way for me to know. The only reason I think it might have been obtained by bot is that the owner of the website in question freely admitted to botting and code injection on his/her blog.
That said, it's probably a good idea to send CCP a petition questioning whether it is alright to go ahead and use it, before I get my ass banned because I was too lazy to ask :P Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
578
|
Posted - 2012.01.19 05:14:00 -
[322] - Quote
Even if it is somehow greenlighted (I personally doubt it) Does it not set a very bad example using data from a site that admits to breaking the EULA?
What do you need to match that data? To avoid you having to use such questionable data in my opinion I am willing to start typing out the data myself. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
74
|
Posted - 2012.01.19 07:25:00 -
[323] - Quote
In this case, I do not think it would be bad to use the data. There are other sites containing the exact same information, where I could've gotten it from too. Besides, I do not think this really constitutes an unfair game advantage, so there is no reason for the data to not be public anyway (in my opinion). Besides CCP time limitations anyway.
Using data from third parties only bothers me if I can think of ways to misuse it. Anything that saves typing and generally decreases the amount of spreadsheet fu involved is a generally a good thing in my opinion.
Anyway, a petition has been sent. Now I just have to wait for the results. Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
594
|
Posted - 2012.01.19 08:30:00 -
[324] - Quote
Well in the meantime I have been typing away for hours here to give you the complete CONCORD LP store data.
Even if you decide to use that data from a botter. I will continue to gather this data so you can replace it with legit and cleanly made data from the community.
I will start with CONCORD obviously.
Then the Faction Navy
Then others.
BTW y u no in game or in your channel so we can talk about this?
Edit: Had a connection issue but I had saved! Whew! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
74
|
Posted - 2012.01.19 16:56:00 -
[325] - Quote
I advise you to wait with writing out the data until I can fabricate a web form. Otherwise I might not be able to parse it. Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
598
|
Posted - 2012.01.20 00:02:00 -
[326] - Quote
Callean Drevus wrote:I advise you to wait with writing out the data until I can fabricate a web form. Otherwise I might not be able to parse it.
That is fine. What I would like to request tho is it be made into a form that I can copy pasta over and over again filling the info. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
74
|
Posted - 2012.01.21 13:22:00 -
[327] - Quote
Ok, got something else updated today. The market scanner page now allows you to choose between history and order uploads (be sure to use the same setting for the market window in your client!), and dynamically shows how well updated a given region is. You can now see your contribution actually mattering again. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
631
|
Posted - 2012.01.21 14:27:00 -
[328] - Quote
I prefer the old "start/stop" button being on the far right edge - made it easier to pause the updating because I could set things up so that the browser was still visible behind the market window. (This matters more on 1024x768 client windows, where space is limited.)
I'm also curious whether you can tone down the frequency of update for the 3.5 sec counter, maybe only update every 0.5 seconds. It now seems to cause graphical lag issues, even when the clients are minimized. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.21 14:39:00 -
[329] - Quote
Have you had the lag issue before? I never heard anything about it before, and it seems to work perfectly here. At least have not noticed any oddities.
Anyway, the start stop button was ugly, but I might add it again next to the group dropdown if it's really a problem. Isn't it still possible to set it up so that it works by just switching the window positions (Market on the right hand side, browser left)? Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
631
|
Posted - 2012.01.21 14:58:00 -
[330] - Quote
I'm not entirely convinced that it's EMK's site that is doing it, I think one of the recent updates to the EVE client borked things. It used to be that when I minimized a market scanning window, the EVE client went from using 80% of a CPU core down to about 20-30% of a single CPU core. Now, the EVE clients are a lot more piggy about using the CPU - so it may be a CCP thing.
As long as the start/stop button is as far left/right as possible in the browser window, I can adjust. The advantage to having it on the right side is that the browser's "close" buttons are then available in cases where the client is laggy and hitting the "stop/pause" button doesn't seem to be working. If I put the browser left and market window to the right, the market window covers up those minimize/close window controls (making it harder to shutdown the IGB if it's laggy).
Also, this started yesterday on the group summary pages: http://www.evemarketeer.com/item/group_summary/78 MYSQL ERROR OCCURRED: Unknown column 'ji0.sell' in 'field list' |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.21 15:03:00 -
[331] - Quote
In that case, I think it must be the client, as the code for the countdown has been like that for over half a year :P
I agree that having the close buttons visible makes more sense. I'll see what I can do in that regard.
The error on the group summary page is probably due to an old bookmark, if you change the decision dropdown to what you want to see (or to and from another) it should be fixed. Developer/Creator of EVE Marketeer
|
6ie
State War Academy Caldari State
0
|
Posted - 2012.01.21 19:14:00 -
[332] - Quote
No graphical issues for me with the count down. I'd prefer it configurable tho - I know it works well at 3s, and gets touchy around 2s.
Is whats selected to be scanned based upon my region setting for evemarketeer.com? (ie do i need to change the region depending on where I am).
Also, Ive seen multiple clients doing exactly the same scan for the same items at the same time before.
I do like the look of the new scanner page. Icons and text are a little large for my tastes. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.21 20:21:00 -
[333] - Quote
What's selected to be scanned is based on what region you are currently in, and what is most outdated in that region. The problem where multiple clients in the same region would upload the same things should have been fixed.
In regards to the size of the images, it is a matter of taste, but there's nothing else to show on that page, so there is no practical reason to make them tiny (and it might have something to do with display resolution :) ?). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
635
|
Posted - 2012.01.24 14:45:00 -
[334] - Quote
The new unified uploader doesn't seem to be reading the cache properly after today's patch.
(And maybe you'll want to break out issues with the Unified Uploader to a new thread?)
Edit: The 2.6 EMD uploader works fine (so will use that today). |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.24 17:21:00 -
[335] - Quote
Have you tried restarting it? :P It should be an issue with the autodetected cache folder, so in the worst case, removing that from the config file and starting the uploader should work. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
637
|
Posted - 2012.01.24 23:36:00 -
[336] - Quote
Callean Drevus wrote:Have you tried restarting it? :P It should be an issue with the autodetected cache folder, so in the worst case, removing that from the config file and starting the uploader should work.
It's working now, but was definitely not working earlier (even after restarting the uploader twice).
I can only guess that I hadn't fired up every single junction mapping that I use (I have 3, and I had only fired up 2 earlier, so 1 of the junctions didn't have the latest "client update"). Or things were extra wonky earlier today.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.25 09:00:00 -
[337] - Quote
Ah, yes, that could be the case. If only one of the clients updates, the uploader will still use the most recent cache folder (in this case, only one client) and thus, all uploads from the older clients will not get registered. Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
615
|
Posted - 2012.01.26 09:18:00 -
[338] - Quote
I am still looking forward to gathering this LP store data when you are ready. |
Omega Flames
Bene Gesserit ChapterHouse Sanctuary Pact
19
|
Posted - 2012.01.26 09:23:00 -
[339] - Quote
Callean Drevus wrote:Ah, yes, that could be the case. If only one of the clients updates, the uploader will still use the most recent cache folder (in this case, only one client) and thus, all uploads from the older clients will not get registered. Was just about to update this with the error im getting now but seems Bob beat me to it, however I'm abit lost as to what I need to do to get it working again. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.26 20:22:00 -
[340] - Quote
Good to hear your problem has been solved.
I've updated the EMK database to use the 1.1 crucible data dump. Developer/Creator of EVE Marketeer
|
|
Scrapyard Bob
EVE University Ivy League
650
|
Posted - 2012.01.27 06:44:00 -
[341] - Quote
http://www.evemarketeer.com/statistics/regions
Is showing strange data in the average age column. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.27 07:53:00 -
[342] - Quote
Yeah, I'd seen it, but I could not come up with a satisfactory reaon for why it was happening, so I deferred fixing it till the weekend Developer/Creator of EVE Marketeer
|
Invictra Atreides
Toward the Terra
27
|
Posted - 2012.01.28 10:44:00 -
[343] - Quote
When uploading could we get more Market groups? When I only want to upload PI goods then I need to make a custom group and there are so many items that it takes forever to create it. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
75
|
Posted - 2012.01.28 11:02:00 -
[344] - Quote
Sure! I just don't know what people generally want, so it's a bit difficult to come up with logical global groups. I'll see if I can add a global PI group.
UPDATE: PI group added. Developer/Creator of EVE Marketeer
|
Invictra Atreides
Toward the Terra
27
|
Posted - 2012.01.28 14:44:00 -
[345] - Quote
Thank you
Quote:I just don't know what people generally want, so it's a bit difficult to come up with logical global groups I understand. So this means that every one has its own item priority and the global groups in the current state cant please every one.
Could this approach be more user friendly:
[ ]All items GööGöÇ[ ]Blueprints GööGöÇ[ ]Drones GööGöÇ[ ]Ship Equipment GööGöÇ[ ]Manufacture & Research ___GööGöÇ[ ]Components ___GööGöÇ[ ]Materials ______GööGöÇ[ ]Ice & Ice Products ______GööGöÇ[ ]ORE & Minerals ______GööGöÇ[*]Planetary Materials
As illustrated above I open any subgroup that I want and mark it with *. All marked groups/items would then be included in the upload process. And a select all and deselect all buttons would also be cool And at the end there could be a Save custom group button to save it under a custom name. Something like this would make it easier to add many items under the same Market group to the custom group.
PS I rly love how your site is growing in visuals and in features. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
76
|
Posted - 2012.01.28 14:56:00 -
[346] - Quote
Yup, that would be easier indeed, but it would also be more intensive in terms of resources and time spent developing it.
As this is the first time someone actually asked for an addition global group, it hasn't been very high on my list of priorities though someone did suggest something like that before.
Anyway, I'll get around to doing it 'sometime'. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
77
|
Posted - 2012.01.28 21:09:00 -
[347] - Quote
Just uploaded unified uploader 0.5, which has support for not deleting cache files, and a few tiny other fixes. I hope you will use another program to delete the cache files though, or the unified uploader is going to eat you disk alive (and upload much slower, because it's ploughing trough thousands of files to find one that is actually useful) :P Developer/Creator of EVE Marketeer
|
6ie
The Kiwis
1
|
Posted - 2012.01.29 01:39:00 -
[348] - Quote
Nice, you rock - thanks! |
FB Tuz
Through the Looking Glass
0
|
Posted - 2012.01.29 10:49:00 -
[349] - Quote
Quite the same... You rock, dude ! Unified uploader 0.5 fixed the KeyError I was having with 0.4b (well technically it still shows in the log, but the uploader runs flawlessly).
Could the market scanner page display what region is currently selected for the upload suggestions ? I often scan while I haul, and I couldn't find at which frequency (or after how many items) the webpage refreshes itself and queries again my location. If at least the detected region was shown, I could refresh manually the page when I see myself in a different region.
Thanks ! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
79
|
Posted - 2012.01.29 10:55:00 -
[350] - Quote
Ah, it adds new items about every 15 items. However, it doesn't really matter whether it is uploading in the exact region it thinks it is uploading.
I could explain it in more detail, but it'd be incomprehensible (believe me, I just tried writing it down, and it failed). Suffice to say that while the upload page thinks you are in a different region, the eve client knows the exact region very well and therefore uploads correct data.
That said, having region detection on the uploader page is a good idea, if just to give you an idea of where you think you are currently uploading. Developer/Creator of EVE Marketeer
|
|
FB Tuz
Through the Looking Glass
0
|
Posted - 2012.01.29 11:37:00 -
[351] - Quote
Yeah, 15 items is nice ! EMD scanner was giving you 100 items at once, which was a tad too many (Kudos to Kallahar though, I only switched to EMK recently - and still have all my scripts API calls to convert...). As for the scanner, it would just help to upload the right old-updated items, but indeed it feeds cache files that are correctly region-stamped (ofc I checked on the uploader log and the region ID is the right one).
I'm really impressed Callean... If only I managed to run through my todo lists and backlogs half as quickly as you, I'd be very happy ! (and my users too :P) |
Vaerah Vahrokha
Vahrokh Consulting
232
|
Posted - 2012.01.29 11:58:00 -
[352] - Quote
I installed the updater, it uploaded stuff once but now it says:
"When patching EVE, CLOSE THIS PROGRAM! Re-run when patch finishes."
too bad I am not patching anything.
Another thing, what's the Upload ID for? I put some random crap and it uploaded prices, did I have to put in something particular instead? Nyx in hi sec: http://img33.imageshack.us/img33/7929/highsecnyxforsale.jpg |
Invictra Atreides
Toward the Terra
27
|
Posted - 2012.01.29 13:45:00 -
[353] - Quote
Vaerah Vahrokha wrote:Another thing, what's the Upload ID for? I put some random crap and it uploaded prices, did I have to put in something particular instead? That might be your username or maybe not. Anyway use the Unified uploader 0.5 istead of 2.6.1EM Uploader.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
79
|
Posted - 2012.01.29 14:31:00 -
[354] - Quote
Lolz, yeah, it sais that every time it starts up, just to remind you :) (the old uploader at least)
I would also reccommend using the unified uploader 0.5, as that one works much better than the previous one.
The upload ID (or key) is the key the website gives you to identify the uploader of the data with. In the case of EVE Marketeer, it is shown on the market scanner page (when logged in). Or in the account menu (also when logged in :P). Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
233
|
Posted - 2012.01.29 16:00:00 -
[355] - Quote
Callean Drevus wrote:Lolz, yeah, it sais that every time it starts up, just to remind you :) (the old uploader at least)
I would also reccommend using the unified uploader 0.5, as that one works much better than the previous one.
The upload ID (or key) is the key the website gives you to identify the uploader of the data with. In the case of EVE Marketeer, it is shown on the market scanner page (when logged in). Or in the account menu (also when logged in :P).
Ah, I registered myself with Firefox, went to the account page and only have seen
"Your upload key is BLAHBLAH" without any mention to the ID.
Since the uploader I used runs outside of EvE, maybe that ID should be visible in the account page with a regular browser.
Also, when I go to the downloads link I see both programs with no mention to "deprecated".
In fact, few days ago a guy actually posted on the MD forum about he using "EVE Marketeer Uploader" for the automated market windows browsing, he did not mention about the Unified updater.
You might want to flag the obsolete software in some way.
Edit:
I find the "delete cache files" option breaks the EULA, because it actively interacts with the client files. I am somewhat perplexed because I know CCP operates with the grace of an elephant in these cases. Reverence seems to be officially approved, is this software CCP checked as well?
Edit 2:
There's a little typo in the Endpoints editor. When you edit the Endpoint data type, the last type reads "Univeral" but I suppose it should read "Universal". Nyx in hi sec: http://img33.imageshack.us/img33/7929/highsecnyxforsale.jpg |
Invictra Atreides
Toward the Terra
27
|
Posted - 2012.01.29 16:45:00 -
[356] - Quote
To stop it you need to click the same icon that you used to for start. Its a Play/Pause icon but the visibility is rly bad while active. If you open the unified uploader then you can see that it is uploading whenever it gets new data. No uploaded data will be lost if you close the Browser, Close the EVE client, throw the PC out of a window. |
Vaerah Vahrokha
Vahrokh Consulting
233
|
Posted - 2012.01.29 17:15:00 -
[357] - Quote
Invictra Atreides wrote:To stop it you need to click the same icon that you used to for start. Its a Play/Pause icon but the visibility is rly bad while active. If you open the unified uploader then you can see that it is uploading whenever it gets new data. No uploaded data will be lost if you close the Browser, Close the EVE client, throw the PC out of a window.
Thank you.
I have another suggestion.
Make the visual cues about the currently selected IGB upload feature (orders vs history) MUCH more evident. The currently selected one should look brigther or something, as of now they look almost the same.
Edit: and another:
Please add a checkbox to not cycle uploads. Expecially history ones. Nyx in hi sec: http://img33.imageshack.us/img33/7929/highsecnyxforsale.jpg |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
79
|
Posted - 2012.01.29 18:11:00 -
[358] - Quote
Ok, let me clear up a few points.
The Unified Uploader is preferred, but not yet fully tested (although functional), that's why I haven't deprecated the other uploader yet.
The 'upload key' is for all intents and purposes identical to the 'upload id'. Any reference to 'upload id' is purely accidental and should be 'upload key'. However, I believe EVE Marketdata calls it Upload ID, and EVE marketeer calls it upload key.
This software uses reverence, and deleting cache files can hardly be seen as a criminal act. It isn't criminal if I delete the EVE client folder either. In any case, as far as I know nobody was ever banned for deleting cache files, and multiple uploaders have had that feature for a very long time.
I'll take a look at the points you mentioned. Developer/Creator of EVE Marketeer
|
Nebu Retski
EVE University Ivy League
6
|
Posted - 2012.01.30 14:07:00 -
[359] - Quote
Regarding the credits awarded for uploading data: Does it only work with the unified uploader or does the IGB also provide credits? I wonder as I have only been uploading using the IGB because I play on Linux and I haven't been able to run the unified uploader yet (haven't done much effort for it though).
Currently my credits balance say's I have 0.00 credits, but at my account info it mentions that I have uploaded cache/logfiles. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
81
|
Posted - 2012.01.30 20:56:00 -
[360] - Quote
You cannot upload with just the IGB. The IGB page merely provides the infrastructure of the cache scraper (Unified Uploader) to do it's work. I think it should be possible to get the unified uploader working on Unix. It isn't a program with a lot of dependencies I think. Developer/Creator of EVE Marketeer
|
|
Scrapyard Bob
EVE University Ivy League
662
|
Posted - 2012.01.31 04:12:00 -
[361] - Quote
In fact, if you run any of the "upload" tools - you can mix and match with any of the "in-game browser market scanner pages" (or write your own page).
For a very long time, I ran the EMD's uploader, but used either EMK or the EveCentral's market scanner page. Or ran the unified uploader, then used any of EMD's / EMK's or EC's market scanning page.
For fun, I tried running the unified uploader under WINE 1.2 on Ubuntu.
Traceback (most recent call last):
File "emkUploader.py", line 5, in (module)
File "zipextimporter.pyo", line 82, in load_module
File "wx\__init__.pyo", line 45, in (module)
File "zipextimporter.pyo", line 82, in load_module
File "wx\_core.pyo", line 4, in (module)
File "zipextimporter.pyo", line 98, in load_module
ImportError: MemoryLoadLibrary failed loading wx\_core_.pyd
(Since I don't run EVE on Linux, I don't plan on troubleshooting this any deeper.) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
81
|
Posted - 2012.01.31 07:01:00 -
[362] - Quote
Aw poop, and here I was thinking it should work fine :P
Back to the drawing board I guess :) Developer/Creator of EVE Marketeer
|
Endeavour Starfleet
Center for Advanced Studies Gallente Federation
624
|
Posted - 2012.01.31 11:06:00 -
[363] - Quote
Any news on the LP store stuff? I am still ready to enter data. |
Vaerah Vahrokha
Vahrokh Consulting
239
|
Posted - 2012.01.31 11:14:00 -
[364] - Quote
I found out some areas of possible usability improvement.
1) Main window => search edit box.
When you have an exact match, the "browse" button should act as "show" and directly show the commodity. As of now when I open the website I come with an item name already filled-in (so, no drop down for completion that directly brings to the commodity sheet). So I have to either delete some part of that name to show the drop down or mistakenly press browse believing the website will look up that item but instead it shows a general search window.
2) Prices exports
By default when you export a market history you get it labelled as "xml.txt", "json", "csv.txt".
The guys using the exports, on the other hand, do it periodically and with method.
Since I clearly see the information gets passed in the URL, what about making a very "batch friendly" export file name like this:
From current
http://www.evemarketeer.com/api/ohlc/10000002/17888/csv
generating a plain "csv.txt" file name in the Save As... window (tried with Firefox only).
To
http://www.evemarketeer.com/api/ohlc/10000002/17888/csv
String splitting that path and activating the following pseudo procedure:
a) Store current date in $CURRENT_DATE. Important: date format must be YYYY-MM-DD b) Lookup 10000002 => $REGION = "The Forge" c) Lookup 17888 => $ITEM_NAME = "Nitrogen Isotopes" d) $EXTENSION = "csv"
Propose end user this file name:
$CURRENT_DATE . $REGION . $ITEM_NAME . '.' . $EXTENSION
Example of resulting file name:
2012-01-31_The Forge_Nitrogen Isotopes.csv
3) Price history duration
I noticed that you can query back to a maximum of 1800 days ago. Do you actually delete data older than that?
Otherwise I'd gladly spend credits / ISK to have the option of downloading the full stored history (hint). Nyx in hi sec: http://img33.imageshack.us/img33/7929/highsecnyxforsale.jpg |
Nebu Retski
EVE University Ivy League
6
|
Posted - 2012.01.31 12:26:00 -
[365] - Quote
Scrapyard Bob wrote:For fun, I tried running the unified uploader under WINE 1.2 on Ubuntu.
WINE 1.2 is really old. You better update to the latest wine version 1.3.3*, it is "not stable" though, but stable enough to do most things. Pretty much "most" people use one of the 1.3.2*+ versions to run EVE. So Callean Drevus, no need to get back to the drawing board yet based on the errors Silent Bob got.
Now I properly tried to run the uploader again and I noticed why it didn't work at all before. After unzipping unifieduploader-0.*.zip, the contents of the lib folder remained zipped. When I unzipped the contents of the lib folder I could run the uploader.exe, however it still doesn't work properly. I get the GUI, but I also get the following error immediatly in the terminal:
Debug: src/helpers.cpp(140): 'CreateActCtx' failed with error 0x00000103 (no more data available).
I went in the config.ini file to add my upload key:
endpoint_0_uploadkey = *************
This however didn't remove the error yet. I also noticed that the path you added for the installation folder seems to be hardcoded to use the windows style (i.e. c:\bla\bla\CCP\EVE\...) while this will not work on *nix type OS's. I'm not sure if wine actually translates it properly or not, but to make code crossplatform compatible there is a function in Python that handles the path style depending on the OS -> os.path
Tonight I will check at home if I can get any further because I then I will have a running eve client. I'll let you know how it goes.
|
Maeste Madeveda
The Spawning Pool Team Liquid
2
|
Posted - 2012.01.31 15:05:00 -
[366] - Quote
Vaerah Vahrokha wrote:2) Prices exports By default when you export a market history you get it labelled as "xml.txt", "json", "csv.txt". The guys using the exports, on the other hand, do it periodically and with method. Since I clearly see the information gets passed in the URL, what about making a very "batch friendly" export file name like this: From current http://www.evemarketeer.com/api/ohlc/10000002/17888/csvgenerating a plain "csv.txt" file name in the Save As... window (tried with Firefox only). To http://www.evemarketeer.com/api/ohlc/10000002/17888/csvString splitting that path and activating the following pseudo procedure: a) Store current date in $CURRENT_DATE. Important: date format must be YYYY-MM-DD b) Lookup 10000002 => $REGION = "The Forge" c) Lookup 17888 => $ITEM_NAME = "Nitrogen Isotopes" d) $EXTENSION = "csv" Propose end user this file name: $CURRENT_DATE . $REGION . $ITEM_NAME . '.' . $EXTENSION Example of resulting file name: 2012-01-31_The Forge_Nitrogen Isotopes.csv I have mixed feelings on this proposition. On one hand it is easier for humans to identify the items so it's nice to have if you look after your exports, on the other hand most of the hardcore data pullers have their own database, their own pulling automaton and so on. Introducing names introduces also tedious coding as in most cases you will be subject to case related bugs.
The date could be a valuable information but you can pull it from the csv in itself as well as using your own system date to add the date, as such it's not a vital information.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
81
|
Posted - 2012.01.31 18:26:00 -
[367] - Quote
@VV
1. The think you propose would not really be an usablity improvement, since there's almost always an exact match in the searchfield. There would be no option to browse anymore if I did that. That said, just clicking the search field, not deleting anything, and pressing enter should do the trick.
If anyone agrees with VV, please tell me, because this may just be my misconception as the one who built it.
2. It should be possible to give the downloaded data a different automatic filename than the URL, if that is possible, this is a very plausible and useful suggestion. Otherwise, because this API is generally used by programmers, I do not think I will change this, since it would require me to do a lot of lookups in tables (for the names) that would make the API slower, and I want that to be as fast as possible.
In that case, it might be possible to duplicate the same functionality with a different call, which shouldn't be a problem.
3. Nah, I don't delete data, but the data goes only back 2 years or so, so I should be safe for a long while yet ;) The full stored history is already available (history API call). If you know a place where I can get history that goes back further, please let me know.
@Nebu Retski: The client cache URL is actually not in the config file at all. When you start the client for the first time, the cache location is automatically determined (and saved to the config). The string you saw was thus the URL that was automatically determined.
I'm very interested to hear whether it works.
@Endeavour Starfleet: Yes! I've gotten a reply to my petition, stating that I am free to use the data (or rather, that they cannot forbid me to use it :P), but that they cannot guarantee the accuracy of it. This means that the nature of the trouble is going to change to checking whether the data is correct. If it is not, we'll have to manually enter the stuff anyway. Since the query to get the information is nearly ready, I'll have this function up soon :) Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
239
|
Posted - 2012.01.31 20:07:00 -
[368] - Quote
Callean Drevus wrote:@VV
1. The think you propose would not really be an usablity improvement, since there's almost always an exact match in the searchfield. There would be no option to browse anymore if I did that. That said, just clicking the search field, not deleting anything, and pressing enter should do the trick.
If anyone agrees with VV, please tell me, because this may just be my misconception as the one who built it.
Steps to reproduce in Firefox latest version in Win 7 64 bit.
1) Start browser => history => reopen last session
or
1a) Add new tab, then go history => reopen the EvE Marketeer page
2) The page shows up with the item already typed in
3) Clicking on it will only select all words of the item name (no drop down)
4) Clicking again will select one word of the item name (no drop down)
Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
81
|
Posted - 2012.01.31 20:41:00 -
[369] - Quote
Yes, I understand what is happening, no trouble there :P
Or was this for other people? Developer/Creator of EVE Marketeer
|
Nebu Retski
EVE University Ivy League
7
|
Posted - 2012.01.31 21:54:00 -
[370] - Quote
Well it seems that it is working (after having the uploader detect my installation folder) when I have a client open and scanning in the IGB.
At least I see the following type of information rolling in the status tab: EVE Central: 2012-01-31 21:47:25: Uploaded 8253 in 10000042 with 3 orders EVE Marketdata: 2012-01-31 21:47:29: Uploaded 31802 in 10000042 with 24 orders EVE Central: 2012-01-31 21:47:29: Uploaded 31802 in 10000042 with 24 orders EVE Marketeer: 2012-01-31 21:47:29: Uploaded 31802 in 10000042 with 24 orders
It also lists a certain number of uploads for the various endpoints etc...
I'll check tomorrow to see if my account received credits for the uploads or not.
There is still however the Debug message I posted earlier that: Debug: src/helpers.cpp(140): 'CreateActCtx' failed with error 0x00000103 (no more data available).
No idea yet why I get that message, but it seems that it is not necessarily related to your uploader program as I have found some posts on the eve forums with the same message but for the eve repair tool.
Edit: Ok my account received credits so it seems to be working fine |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
83
|
Posted - 2012.01.31 22:38:00 -
[371] - Quote
Awesome, happy to hear it's working :)
I also added two new pages to the website. I've found that I gather a lot of data, but not nearly everything is exposed, which is why I am now trying to at least show all the API information I gather.
The first two pages are Industry Jobs, and Wallet Information for corporations (Industry Jobs for characters coming soon). Developer/Creator of EVE Marketeer
|
Steve Ronuken
Fuzzwork Enterprises
203
|
Posted - 2012.01.31 22:57:00 -
[372] - Quote
Great tool :) Appreciate the work you've put in on this.
For people who want to keep an eye on the orders in their areas, and run the unified uploader, this is a very cheap and nasty uploader script, that can be used as an endpoint in it (if you've got an php/mysql server somewhere. like XAMPP). http://www.fuzzwork.co.uk/resources/upload.zip
Easier to manage than multiple uploaders, when you don't want them butting heads, but still want a local copy for analysis. It's cheap and nasty, but the data goes in. FuzzWork Enterprises http://www.fuzzwork.co.uk/ Blueprint calculator and other 'useful' utilities. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
83
|
Posted - 2012.01.31 23:02:00 -
[373] - Quote
Lol, looks quick and nasty indeed. But it works, and that's what important Developer/Creator of EVE Marketeer
|
Steve Ronuken
Fuzzwork Enterprises
203
|
Posted - 2012.01.31 23:05:00 -
[374] - Quote
About the only thing that's not nasty about it is the use of binds with a prepared statement. Might not be validating the datatypes, but no need to worry about escaping. FuzzWork Enterprises http://www.fuzzwork.co.uk/ Blueprint calculator and other 'useful' utilities. |
Maeste Madeveda
The Spawning Pool Team Liquid
2
|
Posted - 2012.02.01 11:39:00 -
[375] - Quote
I stumbled upon a LP store database in my web travels : http://www.ellatha.com/eve/LP-Stores. Could prove useful. |
6ie
The Kiwis
3
|
Posted - 2012.02.01 11:40:00 -
[376] - Quote
To get orders into your own DB - I use EOApp http://www.eveorg.net/eoapp/index.aspx It saves the cache files right into your MYSQL or SQL DB. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
83
|
Posted - 2012.02.01 16:08:00 -
[377] - Quote
Okey. I've added the insanely cool ability to find the items in LP stores that give you the most ISK for your LP. You can select either by sell or buy orders, but obviously the buy orders give you a more senseful result.
I've also added a feature to the item information page where you will be able to see the places where you can get that item in loyalty point stores (if it is actually possible to buy it somewhere).
Oh, and for the people who do not know yet, you can get the links for IGB features when you right click on either item or location links :) Developer/Creator of EVE Marketeer
|
Nebu Retski
EVE University Ivy League
7
|
Posted - 2012.02.01 20:27:00 -
[378] - Quote
So after a bit more of testing I noticed that the uploader always crashes after 270 or 271 uploads. Yesterday I was running the uploader in a 64-bit environment, while today I ran is in a 32-bit environment, but everytime the result is the same. Manually restarting the uploader allows me to provide another 270/271 uploads.
You can find the error log here, although you might not be much with it as a lot of the output is comming from wine.
For completeness sake: I'm running 3.1.2-1-ARCH and wine 1.3.33
A possible solution could be to produce a native linux binary. This most likely requires you to have a linux distribution at hand to compile it or provide the source code to a trustworthy friend that can do it for you. In pretty much a similar way you could produce a native mac version. Anyway that is totally up to you to decide. In the meantime I will keep using it as is because I'm usually in the neighbourhood of my pc when uploading.
Keep up the good work!
|
Gabba Cyno
11
|
Posted - 2012.02.01 20:39:00 -
[379] - Quote
I am having a small amount of trouble displaying history on some PI items in my google docs XLS. I am able to pull all sorts of current info about items but when I try to pull history of the items I run into problems.
=ImportXML("http://www.evemarketeer.com/api/info/2867_2868_2869_2870_2871_2872_2875_2876/xml/10000002";"//sell_lowest5")
This is what I have in a sell that shows me the sell price of the lowest 5% of P4 PI items. That works great. But I am stock on how to display history of these items. I would love to display 30, 60, 90 day info on the following.
average: average transactions a day over this time maximum: maximum transactions a day over this time minimum: minimum transactions a day over this time average_price: average price over this time maximum_price: maximum price over this time minimum_price: minimum price over this time
Any help on what code I need to put in my googe doc cell to make this work? . |
Nebu Retski
EVE University Ivy League
8
|
Posted - 2012.02.01 20:59:00 -
[380] - Quote
=ImportXML("http://www.evemarketeer.com/api/averages/10000043/90/34_35_36/xml";"//average")
This gives me pretty much a column of the average transactions per day for the 3 different items. Now you just make 5 more columns where you change //average with //maximum ...
There might be a better way to do it, but I don't know atm :) |
|
Gabba Cyno
11
|
Posted - 2012.02.01 21:07:00 -
[381] - Quote
Nebu Retski wrote:=ImportXML("http://www.evemarketeer.com/api/averages/10000043/90/34_35_36/xml";"//average")
This gives me pretty much a column of the average transactions per day for the 3 different items. Now you just make 5 more columns where you change //average with //maximum ...
There might be a better way to do it, but I don't know atm :)
Perfect. Thank you. I was missing the //average at the end. . |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.02.02 13:06:00 -
[382] - Quote
Awesome new tool with the Best Loyalty Points functionality. Is there anyway to restrict it to a certain type? ie: caldari navy LPs, or Amarr Navy, LPs, etc? |
Scrapyard Bob
EVE University Ivy League
662
|
Posted - 2012.02.02 13:21:00 -
[383] - Quote
Any ideas / plans yet on adding "virtual" regions to the list of regions? You'd definitely need to do the calculations based only on orders downloaded within the past 7 days. https://forums.eveonline.com/default.aspx?g=posts&m=623346#post623346
I also figured out why the EVE client was being so piggy about the CPU when doing "history" scans. The market window is very inefficient at drawing those history graphs. Switching to "Show Table" view keeps things humming along nicely. |
Scrapyard Bob
EVE University Ivy League
662
|
Posted - 2012.02.02 13:30:00 -
[384] - Quote
Also, the "play" button to start/pause scanning is fine on the left. I've figured out that I can click the "GO" button in the IGB to pause scanning. (Please make sure that you don't auto-restart scanning based on the status of some cookie.)
|
Scrapyard Bob
EVE University Ivy League
662
|
Posted - 2012.02.02 14:20:00 -
[385] - Quote
What is the \Q/ at the top of the screen for?
One interesting statistic that I'm curious about: What systems have the highest trade volume per day, maybe a list of those where the 7-day moving average puts them into the top 25 systems. Repeated for regions too. The Statistics/Market for the regions is interesting, but since there's a ton of "junk" orders for outrageous prices, the "Sell Total" number isn't very accurate.
Maybe add "7-day transaction total" column to the new statistics/market window. Which might give us a better feel for which regions turn over the most ISK/wk. |
6ie
The Kiwis
3
|
Posted - 2012.02.02 15:10:00 -
[386] - Quote
Every time I clicked the \Q/ I got 1mill in my bank account |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
83
|
Posted - 2012.02.02 16:39:00 -
[387] - Quote
6ie wrote:Every time I clicked the \Q/ I got 1mill in my bank account
Damm! That feature was supposed to be VIP only :P
Will reply to other stuff when not on phone. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
83
|
Posted - 2012.02.02 18:27:00 -
[388] - Quote
Quote:Awesome new tool with the Best Loyalty Points functionality. Is there anyway to restrict it to a certain type? ie: caldari navy LPs, or Amarr Navy, LPs, etc? Not yet, but it is on the backlog :P
Quote:Any ideas / plans yet on adding "virtual" regions to the list of regions? You'd definitely need to do the calculations based only on orders downloaded within the past 7 days. I have long been thinking of adding virtual characters, but virtual regions are new. That said, there is no reason not to add them. Are you talking now about arbitrary regions combined, or arbitrary security ratings combined?
As far as I know, the scanner does not auto restart. Have you experienced that behaviour?
Quote:What is the \Q/ at the top of the screen for? The Q at the top of the screen is meant to be a quicklist for items and destinations (stations, regions, etc) that you can quickly click or view. So far I have failed to find a satisfactory format though. You can add some items by clicking on 'add to emk quickbar' on the item information page.
Quote:One interesting statistic that I'm curious about: What systems have the highest trade volume per day, maybe a list of those where the 7-day moving average puts them into the top 25 systems. Repeated for regions too. The Statistics/Market for the regions is interesting, but since there's a ton of "junk" orders for outrageous prices, the "Sell Total" number isn't very accurate. And that's with a lot of junk orders already filtered out. It's too bad I can't filter items for which there isn't anything else on the market for reasonable prices. A 7 day moving average for transactions seems like a very plausible way to check market volume. Will do!
Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
663
|
Posted - 2012.02.02 22:56:00 -
[389] - Quote
Callean Drevus wrote:Quote:Awesome new tool with the Best Loyalty Points functionality. Is there anyway to restrict it to a certain type? ie: caldari navy LPs, or Amarr Navy, LPs, etc? Not yet, but it is on the backlog :P Quote:Any ideas / plans yet on adding "virtual" regions to the list of regions? You'd definitely need to do the calculations based only on orders downloaded within the past 7 days. I have long been thinking of adding virtual characters, but virtual regions are new. That said, there is no reason not to add them. Are you talking now about arbitrary regions combined, or arbitrary security ratings combined?
Both. But not completely arbitrary, just some major pre-defined groupings. Such as the "hi-sec" price or "market hubs" price.
For instance, on EMD, I can get a price for a Naga across all of hi-sec: http://eve-marketdata.com/price_check.php?step=Show&type_id=4306&type=sell®ion_id=-2
Or just lo-sec: http://eve-marketdata.com/price_check.php?step=Show&type_id=4306&type=sell®ion_id=-3
Or both: http://eve-marketdata.com/price_check.php?step=Show&type_id=4306&type=sell®ion_id=-4
EMD doesn't currently offer a "market hub" option. And it doesn't filter out orders which haven't been updated or downloaded in the past few weeks, so sometimes you get a page of useless information.
Being able to pull an API value for a particular item in a virtual region composed of "all hi-sec, all lo-sec, empire, or null, or market hubs" would be useful as a way of getting a closer valuation of a particular item then just going with the Jita price. While Jita's sell prices are usually the lowest, if there is market manipulation occurring, Jita sell prices can get very whacky. In which case, the long-term value is probably closer to the average of the other (3) market hubs.
So by doing the 5% simulated buy across all (4) market regions or all of hi-sec, I'm hoping that it would give better values for tools like jEvEAssets, EvEHQ, EvEMeep or spreadsheets. Better option then relying on just Jita pricing.
I think a good list of virtual regions would be:
"hi-sec" - any hi-sec station gets included in this price "lo-sec" - same, but for lo-sec stations "null-sec" - same, but for null-sec "empire" - include both hi-sec and low-sec stations "main market hubs" - roll-up Jita, Amarr, Dodixie and Rens "main market regions" - hi-sec stations in Domain, Forge, Heimatar and Sinq Laison
I'd suggest filtering out any orders not updated within the past 7 or 14 days, no sense handing out bad information in these roll-up "virtual" regions. |
Scrapyard Bob
EVE University Ivy League
663
|
Posted - 2012.02.02 22:59:00 -
[390] - Quote
EMD's market scanner page would sometimes auto-restart, which is why I didn't consider that a viable method of pausing the EMD's market scanning page.
EMK's market scanner page does not automatically restart. This is a good thing.
(Yes, I could close the in-game browser window, but that little "x" can be difficult to hit if the client starts lagging. The "Go" button is a lot larger, and the play/pause button is even bigger.) |
|
Gabba Cyno
11
|
Posted - 2012.02.03 05:03:00 -
[391] - Quote
How many IDs can be pulled per importxml? . |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
84
|
Posted - 2012.02.03 06:54:00 -
[392] - Quote
Gabba Cyno wrote:How many IDs can be pulled per importxml? Here I was thinking you people were all idiots because this information was on the API page, but it's not, so I apologize. Anyway, the limit is 25 per call, and I'll soon add that to the page. Developer/Creator of EVE Marketeer
|
Nebu Retski
EVE University Ivy League
9
|
Posted - 2012.02.03 11:11:00 -
[393] - Quote
Callean Drevus wrote:Anyway, the limit is 25 per call, and I'll soon add that to the page.
From some testing that I did, it seems that there might be a bug on your side. At a certain point the return data is not complete. Whenever I request 16 items I only get the first 15 returned (from 1 item until 15 items everything looks fine though), check for yourself -> http://www.evemarketeer.com/api/info/36_35_34_2268_2305_2267_2288_2287_2307_2272_2309_2073_2310_2270_2306_2307/xml/10000043/sell_lowest5
When requesting 18 or more items, the last 2 items will not be returned. 18 items url -> http://www.evemarketeer.com/api/info/36_35_34_2268_2305_2267_2288_2287_2307_2272_2309_2073_2310_2270_2306_2307_2308_2309/xml/10000043/sell_lowest5
25 (should be the hard limit) items url -> http://www.evemarketeer.com/api/info/36_35_34_2268_2305_2267_2288_2287_2307_2272_2309_2073_2310_2270_2306_2307_2308_2309_2393_2396_3779_2401_2390_2397_3683/xml/10000043/sell_lowest5
Can you confirm this bug or else I should scratch my head a bit more as to why it happens to me :-)
|
Scrapyard Bob
EVE University Ivy League
663
|
Posted - 2012.02.03 11:45:00 -
[394] - Quote
(chuckle) Oh look, I'm not the only one who can break things!
\Q/
In that 3rd URL, in the sequence "2308_2309_2393", the 2309 is being dropped from the result set. http://www.evemarketeer.com/item/info/2309
Also in the sequence: 2306_2307_2308, the 2307 is missing http://www.evemarketeer.com/item/info/2307
Region #43 is Domain.
Now the odd thing is that I have almost the exact same query in one of my sheets, but I'm only pulling 24 at once, and #2307 shows up fine for me (along with 2309).
http://www.evemarketeer.com/api/info/2268_2305_2267_2288_2287_2307_2272_2309_2073_2310_2270_2306_2286_2311_2308_2393_2396_3779_2401_2390_2397_2392_3683_2389/xml/10000043/sell_lowest5 |
Nebu Retski
EVE University Ivy League
9
|
Posted - 2012.02.03 12:13:00 -
[395] - Quote
I found what the issue was, so it's not a bug (well it was a little bit, but on my side :-)). You can't have duplicates in your urls (as I had in my url 2*2309 and 2*2307). Once I had removed the duplicate entries I was able to pull 25 in one go.
Thx Bob for pointing it out. |
6ie
The Kiwis
3
|
Posted - 2012.02.03 12:26:00 -
[396] - Quote
The search that was changed in the last few days doesn't work so well.
Cut/Paste in: Medium Anti-Kinetic Screen Reinforcer I then you get a drop down of 4 items click "Medium Anti-Kinetic Screen Reinforcer I" and it takes you to a page: Could not find 'Medium Anti-Kinetic Screen Reinforcer I' Perhaps you meant one of these items? (and list the same 4 items...)
I clicked "Medium Anti-Kinetic Screen Reinforcer I" on the drop down, it knows what i wanted, it should have taken me to the "Medium Anti-Kinetic Screen Reinforcer I" page. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
86
|
Posted - 2012.02.03 20:13:00 -
[397] - Quote
@6ie: Absolutely correct :) I built in the second page to allow people to type in arbitrary things like 'caldari carrier' and still have the option of finding whatever they were looking for. The old search was pretty restrictive (though less restrictive than other sites :P).
I agree however, that when you click the dropdown, it should take you to the item immediately. Will fix that.
@API trouble: Ah, yes. The server doesn't like it when you query for the same thing twice. Or rather, it doesn't bother returning it twice. If you really need that functionality it may be better to just query them all independently. Developer/Creator of EVE Marketeer
|
Jaqa
Screaming War Eagles Incorporated
2
|
Posted - 2012.02.04 22:54:00 -
[398] - Quote
Excellent site--keep up the great work. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
88
|
Posted - 2012.02.05 08:20:00 -
[399] - Quote
Thank you.
I added PI information (for PI items) and filtering for the best loyalty point items this weekend (and some small things which I forgot). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
672
|
Posted - 2012.02.05 11:20:00 -
[400] - Quote
When I was trying out the Loyalty point stuff last night... (loving the feature, BTW).
- It proved to be impossible, once you picked a faction and then changed the corp to a specific one within the faction to get the corp drop down to go back to "All" without changing factions and then going back to the same faction.
1. Pick faction 2. Pick corp 3. decide you want to go back to seeing "all" corps, change back to "all" 4. corp-drop down keeps resetting to list the last corp you looked at after submit
(So there's definitely a problem in both the corp/faction pull-downs where you can't get them back to "all".)
- When looking at an item details, the "LP Store" tab was always blank, but only for some items. Some worked, some didn't.
http://www.evemarketeer.com/item/info/27187 http://www.evemarketeer.com/item/info/27231
- Value column in the bestloyaltypoints page may need an option to work off of the 5-day or 7-day or 30-day moving average historical price rather then sell/buy.
http://www.evemarketeer.com/item/info/27231 (This item seems to only ever change hands for 900k to 2.1M ISK, but the 5% number in the Forge is 15M ISK.)
- Because of the length of names on things like Implants, the mouse-over tool-tip text for the Type column links should probably be changed from "View EMK item information" to the name of the page that you are linking to. But at least the columns are resizable.
|
|
Omega Flames
Bene Gesserit ChapterHouse Sanctuary Pact
20
|
Posted - 2012.02.05 15:38:00 -
[401] - Quote
the item isogen has a buy entry in Uitra for 346,982 isk p/u with a quantity of 0. Even if someone was doing a margin trading scam (which they aren't cause that order does not exist ingame, unless I'm blind which is possible I guess) shouldn't the fact it's a quantity of 0 mean it gets ignored? Cause it's really fubar'ing my prices in evehq :( |
Desmont McCallock
121
|
Posted - 2012.02.05 17:16:00 -
[402] - Quote
Looking at the EVE Tools section, here are some missing info regarding the dev behind certain tools.
EVEMon : The EVEMon Dev Team EFT: Gripen EVSCO: Karbowiak jEveAssets: Golden Gnu Out of EVE: Azazel Mordred aka ShrimpWorks Griefwatch: BattleClinic
BTW the links are broken (i.e. EVEMon http://www.evemarketeer.com/home/tools/16). |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
88
|
Posted - 2012.02.05 19:51:00 -
[403] - Quote
Scrapyard Bob wrote:Lots of stuff... Fixed it all :P Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
672
|
Posted - 2012.02.05 20:38:00 -
[404] - Quote
30-day average price drop-down on the LP page, doesn't seem to actually work. It's still displaying the sell.
1. Change drop down to "Sell Price" 2. Page refreshes, it goes back to saying "Average Price (30d)"
1. Change drop down to "Buy Price" 2. Page refreshes, value change, but drop-list goes back to saying "Average Price (30d)"
|
Vaerah Vahrokha
Vahrokh Consulting
250
|
Posted - 2012.02.05 22:00:00 -
[405] - Quote
There's one feature that I heavily used in the past, on another similar and awesome website: an items basket.
This let me create what is basically the equivalent of the in game market quickbar. I could then switch easily between my most traded items without hopping around pages and typing dozens of item names.
Imagine how cool it'd be if I could just have a clickable list to the left that updates a pane to the right to show the prices and the graphs like the in game market window.
I apologize in advance in case this feature is already implemented and I grandiously missed it. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Omega Flames
Bene Gesserit ChapterHouse Sanctuary Pact
20
|
Posted - 2012.02.05 22:57:00 -
[406] - Quote
Vaerah Vahrokha wrote:There's one feature that I heavily used in the past, on another similar and awesome website: an items basket.
This let me create what is basically the equivalent of the in game market quickbar. I could then switch easily between my most traded items without hopping around pages and typing dozens of item names.
Imagine how cool it'd be if I could just have a clickable list to the left that updates a pane to the right to show the prices and the graphs like the in game market window.
I apologize in advance in case this feature is already implemented and I grandiously missed it. go to an item's page>>>click add to EMK quickbar>>>mouse over the "item" link at the top left of the page |
Vaerah Vahrokha
Vahrokh Consulting
250
|
Posted - 2012.02.06 01:25:00 -
[407] - Quote
Omega Flames wrote:Vaerah Vahrokha wrote:There's one feature that I heavily used in the past, on another similar and awesome website: an items basket.
This let me create what is basically the equivalent of the in game market quickbar. I could then switch easily between my most traded items without hopping around pages and typing dozens of item names.
Imagine how cool it'd be if I could just have a clickable list to the left that updates a pane to the right to show the prices and the graphs like the in game market window.
I apologize in advance in case this feature is already implemented and I grandiously missed it. go to an item's page>>>click add to EMK quickbar>>>mouse over the "item" link at the top left of the page
Ah, I both noticed the EMK quickbar and the item link but once I have pressed add to EMK quickbar and have seen that nothing happened, I thought the functionality was still not implemented.
I loaded another item and now it shows, thank you.
Edit:
AAAARGHHH!!!!
I added PLEX on quickbar and everything exploded! (Is it because it has apostrophes and PHP did not escape them?)
MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's%' AND it.typeName LIKE '%License%' AND it.typeName LIKE '%Extension%' AND it.t' at line 1 MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's%' AND CONCAT(mgp3.marketGroupName, ' ', mgp2.marketGroupName, ' ', mgp.marketG' at line 1
And at the center of the screen:
Could not find '30 Day Pilot's License Extension (PLEX)'
We could not find any items like the one you were searching for.
But this is the very thing the website proposed me in a list, when I entered PLEX in the search box. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Steve Ronuken
Fuzzwork Enterprises
215
|
Posted - 2012.02.06 10:43:00 -
[408] - Quote
Bind variables.
They save you from having to escape everything. They're a /little/ more complex to use, and you can't bind an inlist (well, I've never found out how) but they're seriously worth it. And they can speed things up a tiny amount, if you're also using a database that can cache its compiled sql (as you're using the same sql each time, with different parameters being filled in)
PDO makes them available. FuzzWork Enterprises http://www.fuzzwork.co.uk/ Blueprint calculator and other 'useful' utilities. |
Desmont McCallock
121
|
Posted - 2012.02.06 11:27:00 -
[409] - Quote
"Add to EMK Quickbar" is pretty nice but once that item is added, when showing this item again, the link should switch to "Remove from EMK Quickbar". |
Scrapyard Bob
EVE University Ivy League
672
|
Posted - 2012.02.06 15:33:00 -
[410] - Quote
http://www.evemarketeer.com/item/info/27102 - The LP Stores tab lists all LP items, not just entries for the item being viewed
|
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
88
|
Posted - 2012.02.06 21:21:00 -
[411] - Quote
Ok, let me see:
Quote:30-day average price drop-down on the LP page, doesn't seem to actually work. It's still displaying the sell. Yes, it was not correctly registering what was selected. Fixed it now.
Quote:There's one feature that I heavily used in the past, on another similar and awesome website: an items basket. Would be useful, will see if I can combine the item info page and the quicklist to do something awesome if your screen is wide enough, since quicklist isn't actually quite as fast as the quicklist you propose.
In addition, yes, the quicklist should be updated directly, but I haven't had a chance to do that yet.
Quote:I added PLEX on quickbar and everything exploded! Ehm, yes, it seems there was a problem with escaping in my search alghoritm, I apologize for any distress it caused you, but I assure you it didn't have anything to do with the quicklist :P
I use them almost everywhere, except in the really complex queries where my brain absolutely needs to see what I'm referencing.
Quote:"Add to EMK Quickbar" is pretty nice but once that item is added, when showing this item again, the link should switch to "Remove from EMK Quickbar". Agreed, done.
QUOTE: The LP Stores tab lists all LP items, not just entries for the item being viewed
It just looks like there are a LOT of locations where you can obtain that item to me, no different items at all (all LP and ISK cost is the same, the only thing that differs is faction, corp and location). Can you screenshot this one if you really think there's something wrong? Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
672
|
Posted - 2012.02.07 01:14:00 -
[412] - Quote
Out of curiosity, do we get more credits for uploading history for outdated regions? Or do we always get the same points? |
Scrapyard Bob
EVE University Ivy League
672
|
Posted - 2012.02.07 01:27:00 -
[413] - Quote
Callean Drevus wrote: QUOTE: The LP Stores tab lists all LP items, not just entries for the item being viewed
It just looks like there are a LOT of locations where you can obtain that item to me, no different items at all (all LP and ISK cost is the same, the only thing that differs is faction, corp and location). Can you screenshot this one if you really think there's something wrong?
Ah, okay, I see. The mass of data made me avoid looking at it in detail because it was so repetitive. But since prices never vary at all across different stations for a particular corp it may be better to:
- Drop the Station column. - Make the corporation name a hyper-link to information about that corporation. - Add another page somewhere else detailing the corporation information.
Which would shorten up that list a lot. When I was looking at that tab, my interest is more along the lines of "I want this, which NPC corps sell it?", not "exactly which stations sell it". So if I want to earn a specific implant, what corps would I have to go grind for?
On the "NPC corporation information" page, you could then have tabs for:
- Stations (with Region / Constellation / System / Station / Jumps Away) - Agents (Type / Level / Region / Constellation / System / Station / Jumps Away) - Locator agents (Region / Constellation / System / Station / Level / Jumps Away)
Some of the above is optional and would basically be groundwork for an "agent finder" type tool. Jumps away would be a useful feature to pull the current location from the IGB.
Now, the natural downside is that you would not be able to have one big list of sortable station names to figure out where a specific implant is found. But I think that if you've earned enough LP with a particular corp by doing missions, then you should already have a fairly good idea of where they are located. And you would probably be looking at your Journal to see which corps you have enough LP earned with, which is also a corporation-central view leading to the next question of "where does this corporation have offices and how far away are they". |
Scrapyard Bob
EVE University Ivy League
672
|
Posted - 2012.02.07 01:54:00 -
[414] - Quote
Oops, it broke:
Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38 Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed |
Dragonaire
Corax. The Big Dirty
31
|
Posted - 2012.02.07 05:06:00 -
[415] - Quote
Yeah I'm getting the same error on some pages now too Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
89
|
Posted - 2012.02.07 06:54:00 -
[416] - Quote
Solved now? If not, what are the pages? Developer/Creator of EVE Marketeer
|
6ie
The Kiwis
3
|
Posted - 2012.02.07 13:12:00 -
[417] - Quote
I am getting it sporadically all over the show. Even inside windows. |
Dragonaire
Corax. The Big Dirty
31
|
Posted - 2012.02.07 16:35:00 -
[418] - Quote
I'm guessing from the error message you aren't checking you got an actual database connection before trying to use it. You need to check if it is returning FALSE when it can't get the connection and handle it. I also have had it on multiple different pages when I was looking at stuff yesterday but most of them would display correctly without errors if I refreshed them in the browser. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
89
|
Posted - 2012.02.07 17:36:00 -
[419] - Quote
Correct, I'm not catching this kind of stuff. The most important problem is that it keeps the website from functioning however.
It seems something was eating up mysql connections, thats my first guess anyway. Developer/Creator of EVE Marketeer
|
Jacob Lyon Chieve
Sanguis Mortem Industrial
1
|
Posted - 2012.02.07 17:44:00 -
[420] - Quote
Seems to be working again for me. Was down all morning, but seems functional now.
I was worried I got black-listed for too many XML calls! |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
89
|
Posted - 2012.02.07 22:22:00 -
[421] - Quote
Nope, not blacklisted. Seems the problem had something to do with InnoDB. Stupid innodb size... I hope it has been fixed for now. Developer/Creator of EVE Marketeer
|
Jacob Lyon Chieve
Sanguis Mortem Industrial
1
|
Posted - 2012.02.08 04:23:00 -
[422] - Quote
Having issues once more. On mouseover, the cell containing the XML call says "error: the XPath query did not return any data"
FWIW, I can get onto the website just fine. |
Dragonaire
Corax. The Big Dirty
32
|
Posted - 2012.02.08 05:01:00 -
[423] - Quote
If you're running out of database connections then you have something not releasing them correctly some where. Make sure when you're done with the connection on a page you set you variable for your connection to NULL should help make sure it gets released. Something like $mysql->close();$mysql=NULL; works wonders at times Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Aineko Macx
Royal Amarr Institute Amarr Empire
137
|
Posted - 2012.02.08 06:53:00 -
[424] - Quote
Some random comments/questions: - in the API description for the historical averages, the description for 'averages', 'maximum' and 'minimum' are wrong. Those refer to the item volume, not transaction count. - When querying using station_info, in the returned document station_id is null - How are you estimating the sell/buy percentages? - I'd very much like to have the historical data expanded to have a row per day, instead of just one averaged for the whole period. Would you consider doing that?
Keep up the good work! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
89
|
Posted - 2012.02.08 09:02:00 -
[425] - Quote
Jacob Lyon Chieve wrote:Having issues once more. On mouseover, the cell containing the XML call says "error: the XPath query did not return any data"
FWIW, I can get onto the website just fine.
This might be because Google caches the request your xml call makes for a while, so it remembers whatever happened when the site was down. If the call you are doing by XML is going wrong if you enter the URL in a browser, please give me that URL, so I can investigate.
Quote:If you're running out of database connections then you have something not releasing them correctly some where. Make sure when you're done with the connection on a page you set you variable for your connection to NULL should help make sure it gets released. Something like $mysql->close();$mysql=NULL; works wonders at times Ah, turns out it wasn't connections, but InnoDB log file size, the odd thing is that it continued to work until it really couldn't do anything anymore. I do not exactly understand the reason, but the size has been increased and I do not expect to see this problem again.
- in the API description for the historical averages, the description for 'averages', 'maximum' and 'minimum' are wrong. Those refer to the item volume, not transaction count.
I think you are quite correct. Number of transactions is a meaningless number anyway. I'll fix this to state what it means correctly.
- When querying using station_info, in the returned document station_id is null
Ah, yes, this is an unfortunate artifact of not saving the station id when orders are added together. This will probably change some day, but as you already know what station it is when querying, it isn't exactly high on my list of priorities.
- How are you estimating the sell/buy percentages?
Using magic! No, the calculation is simply to see whether the average lies more in the direction of the highest transaction that day, or the lowest transaction that day. High has a good chance of being sell, low has a good chance of being buy.
- I'd very much like to have the historical data expanded to have a row per day, instead of just one averaged for the whole period. Would you consider doing that?
I thought this was already possible! Where the hell has my history API gone... Anyway, will be added ASAP. UPDATE: Yes! I already had it, but it's not on the API page: http://www.evemarketeer.com/api/history/24698/10000002/xml Developer/Creator of EVE Marketeer
|
Aineko Macx
Royal Amarr Institute Amarr Empire
137
|
Posted - 2012.02.08 19:49:00 -
[426] - Quote
Callean Drevus wrote:- I'd very much like to have the historical data expanded to have a row per day, instead of just one averaged for the whole period. Would you consider doing that? I thought this was already possible! Where the hell has my history API gone... Anyway, will be added ASAP. UPDATE: Yes! I already had it, but it's not on the API page: http://www.evemarketeer.com/api/history/24698/10000002/xml Awesome! I see you are returning the whole history set there. You might want to introduce a time period parameter (today minus n days) so we can query for just the needed history without making your servers angry |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
91
|
Posted - 2012.02.08 21:37:00 -
[427] - Quote
Filtering the days will probably not make it much faster to load, but it will make it easier on you to use ;) I'll add that to the list of stuff to do :P
In addition, I've added support for character groups today, which will allow me to do some very cool things that some other market programs already do. For more information, see the newsitem on the website. And please report any issues you find. Developer/Creator of EVE Marketeer
|
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.02.09 01:35:00 -
[428] - Quote
Getting a lot of Quote:Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38 Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed
Got it both trying to access the index page, as well as the regions page. You messing around with something at the moment? |
Dragonaire
Corax. The Big Dirty
32
|
Posted - 2012.02.09 06:00:00 -
[429] - Quote
Just to go with the above: EVE Marketeer: Upload unknown problem: Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38
Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed
So it's also effecting uploads now Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Scrapyard Bob
EVE University Ivy League
673
|
Posted - 2012.02.09 06:42:00 -
[430] - Quote
Dragonaire wrote:So it's also effecting uploads now
Days like these, I just point my in-game browser at either eve-marketdata market scanner page or eve-central's upload suggestions page.
|
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
92
|
Posted - 2012.02.09 07:28:00 -
[431] - Quote
Damn, the reproduction of the problem is bothering me. I cannot. I'll try to enter a nice new check in the page to see exactly what error is occuring. Seems it had little to do with the mass of errors I fixed in the mysql log.
I suspect my changes to yapeal are to blame. Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. The Big Dirty
32
|
Posted - 2012.02.09 08:24:00 -
[432] - Quote
If it's something with Yapeal contact me and I'll go over it with you to figure it out. Also I've been cleaning some stuff up that might have been causing some slow downs in Yapeal in the last couple days so you might want to look at updating as well. I pushed it to the hg already just haven't updated the archives yet that gets to wait until tomorrow I need to head to bed Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
6ie
The Kiwis
3
|
Posted - 2012.02.09 12:31:00 -
[433] - Quote
Did you fix the search? Its acting weird - put in Projectile Weapon Rigging You get a page: Could not find 'Projectile Weapon Rigging' Perhaps you meant one of these items?
And it only lists one item... Projectile Weapon Rigging
|
Scrapyard Bob
EVE University Ivy League
677
|
Posted - 2012.02.10 16:20:00 -
[434] - Quote
Database errors are still happening on a regular basis, both in the uploads queue as well as various pages on the website.
At a rough guess, about 1/4 of all uploads are failing, and 10-20% of the web page hits. |
Zifrian
Licentia Ex Vereor Intrepid Crossing
208
|
Posted - 2012.02.10 17:52:00 -
[435] - Quote
I've noticed that some people are having this same error:
Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38 Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed
I'm getting this with the API, sometimes it works fine and other times it doesn't.
I added this to EVE IPH a few weeks ago to query the API for average price data. I'm not sure if there is an increase in queries? Let me know if I can help. Maximze your Industry Potential! - Get EVE Isk per Hour! |
Scrapyard Bob
EVE University Ivy League
677
|
Posted - 2012.02.10 19:23:00 -
[436] - Quote
From the uploader over the past few days:
EVE Marketeer - 51.6k EVE MarketData - 68.4k
So EVEMarketeer, which usually runs neck and neck with EMD, is losing about 25% of the upload attempts due to the database error.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
94
|
Posted - 2012.02.10 19:49:00 -
[437] - Quote
Fixing it now
UPDATE: Should be fixed now. Will continue to monitor over the weekend. Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. The Big Dirty
32
|
Posted - 2012.02.11 05:33:00 -
[438] - Quote
Now I've got to know what you were trying to change in Yapeal that caused the problem but I'll let you do it in a Eve-mail if you want instead of here Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Scrapyard Bob
EVE University Ivy League
677
|
Posted - 2012.02.11 06:57:00 -
[439] - Quote
Another wish item for the LP "best loyalty points".
- Sort the two drop-lists in alphabetical order before presenting it to the client.
And another argument in favor of only showing the list of corps that sell a particular LP item rather then every single station.
http://www.evemarketeer.com/item/info/9956
3393 rows returned on the LP Stores tab for that item, which takes a bit to load. :) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
94
|
Posted - 2012.02.11 08:24:00 -
[440] - Quote
Oh, yes, sorting should be something that is easliy implemented, but very handy to have.
Regarding the LP store items: I agree with you that it should be done on a different corporation page (listing all the stations), the loading is not the issue for me though :P just that 3300 items are not very reasy to browse :) Developer/Creator of EVE Marketeer
|
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
96
|
Posted - 2012.02.12 14:19:00 -
[441] - Quote
Ok, today, fixed the manufacturing page, which had been unreliable for quite some time. I dropped recursive building for the moment, since it made the results look pretty odd. You now see what you see ingame as well. It also takes into account a few edge cases, most notably recycled items and extra materials which also show up in the base materials. This is not as much a feature, as an overdue fix. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
267
|
Posted - 2012.02.13 11:12:00 -
[442] - Quote
Very minimal issue but I really wish to see your site to be uber super duper extra perfect pro!
Basically you put the Z-order (at least for Firefox 10.1) of the mouse over tool tips in the main page above the Z-order of the items found drop down.
Screen shot with red arrow showing it.
The tooltips that go over the drop down are not limited to the one you see, if you hover the mouse over other headings they also appear above it. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Vaerah Vahrokha
Vahrokh Consulting
267
|
Posted - 2012.02.13 11:21:00 -
[443] - Quote
Another, less minimal issue I just found.
I entered plex then pressed GO and the website displayed a Could not find 'plex' screen. So far so good.
But now I entered plex in the top search box again, I correctly get the drop down choice but then if I click on the "30 days blah blah blah" entry:
- I expect to go in the PLEX page like it did in the past
- Instead nothing happens and the top search box gets filled by a "29668" number, which I suspect is the item ID. And no PLEX page shown. Now I can try typing "plex" all day long and I only get the number. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
6ie
The Kiwis
3
|
Posted - 2012.02.13 13:34:00 -
[444] - Quote
The search makes absolutely no sense.
Search: "mega af" and press enter or Go Returns 50+ items, not in alphabetical, or in "most likely" order, most of which do not even have an "af" in their title. "Mega Afocal Pulse Maser I" is 12 lines down, almost off the screen.
How it should work: http://eve-marketdata.com/ Search: "mega af" Drop down of 2 items (the only 2 with "mega af" in their title), allowing you to click the one you want, and it takes you right to that items page.
Even if you get the drop down on EVE Marketeer, and click the item you want - it doesn't take you anywhere.
Oh, and GL going back to modify the text in the search box. |
Scrapyard Bob
EVE University Ivy League
682
|
Posted - 2012.02.13 14:27:00 -
[445] - Quote
Vaerah Vahrokha wrote: - Instead nothing happens and the top search box gets filled by a "29668" number, which I suspect is the item ID. And no PLEX page shown. Now I can try typing "plex" all day long and I only get the number.
Yes, this seems to be a new bug that I hit last night on some other items. However, the search worked if I clicked the "go" button and the ItemID only ended up in the search box if I clicked on the "suggestions" list that shows up under the search box as you type stuff in.
Maybe the search box needs to be smart enough to let us search on item ID numbers? :) |
Aineko Macx
Royal Amarr Institute Amarr Empire
139
|
Posted - 2012.02.13 14:32:00 -
[446] - Quote
Having played with the API in the last days, more feedback:
- For computing buyHighest5% and sellLowest5% you should do some pre-filtering on the orders. Right now, the numbers are often biased by existing but irrelevant orders like the token region-wide buy order for 1 millions units at 0,01isk. There are many ways to do that. A simplistic approach would be to discard buy orders below say -80% of the highest buy order, or something like the median of orders whose remaining volume is smaller than the initial volume and are not older than x.
- When using the station_info API, the datetime column is missing (it's there for the otherwise identical info API).
- I'd really like to a) query up 25 typeIDs in the history API and b) specify the time range. I'd rather do 210 queries that way instead of the current 5247 when I update the history |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
98
|
Posted - 2012.02.13 20:47:00 -
[447] - Quote
- Basically you put the Z-order (at least for Firefox 10.1) of the mouse over tool tips in the main page above the Z-order of the items found drop down.
Ah! Correct, I'll fix this soon.
- I entered plex then pressed GO and the website displayed a Could not find 'plex' screen. So far so good.
Umm, yes, it seems I started working on search, then forgot I was doing so, and started implementing different things. I'll fis these issues tonight.
- Maybe the search box needs to be smart enough to let us search on item ID numbers? :)
It should be, unless I broke it. If so, will fix it now :P
- For computing buyHighest5% and sellLowest5% you should do some pre-filtering on the orders. Right now, the numbers are often biased by existing but irrelevant orders like the token region-wide buy order for 1 millions units at 0,01isk. There are many ways to do that. A simplistic approach would be to discard buy orders below say -80% of the highest buy order, or something like the median of orders whose remaining volume is smaller than the initial volume and are not older than x.
There should be a lot of filtering done even before the orders actually get into the database. I've seen indications before that this wasn't correctly happening, but since I haven't received any reports on it thus far I've let it be. Will put checking this on the agenda for asap. A mechanism much like you describe has been implemented, using the highest buy order (or lowest well) as a measure is a smart thing though, and I'll see if I can add that.
- When using the station_info API, the datetime column is missing (it's there for the otherwise identical info API).
Hmm, yes, this seems somewhat odd. I'll see if I can't add this in an easy way (due to the calculations being done on orders it's not automatically and representatively there).
- I'd really like to a) query up 25 typeIDs in the history API and b) specify the time range. I'd rather do 210 small queries that way instead of the current 5247 when I update the history, pulling like 300MB of csv data in the process
a is probably not going to happen, as the history queries are quite intensive as it is (biggest table in emk). b is on my list for implementation (well, limit, but date range is probably even better) :)
- You are tracking more items in the DB than just the subset WHERE marketGroupID IS NOT NULL AND published = 1. Why?
I shouldn't be, it should even be a little less, as I'm not interested in tech 2 BPO's. Where are you getting this number?
The search makes absolutely no sense.
It actually does, it's just smarter than you, and tries it's best to find things with 'mega' and 'af' in it. It didn't give any preferential treatment to the first time though, nor to both items following eachother, which would be a good addition. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
271
|
Posted - 2012.02.13 23:42:00 -
[448] - Quote
1) Thanks for all of this effort you are pouring over.
2) If you ever put additional prices sanitizing please make them optional.
There are certain "price spikes" called Pin Bars that are fundamental to every market, they look like "errors" but they are just the buyers / sellers taking liquidity before pushing hard.
Here's an example of PBs of a real market (gold), I marked them with red arrows.
Here is your own website, correctly showing a prominent pin bar for the PLEX market, seen with monthly bars.
Finally, here's your website again, correctly showing another pin bar for the PLEX market, this time with weekly bars.
These elements (expecially on the daily bars) can very easily get lost if you cut data too much and then the graphs become completely useless.
3) A low priority tip: please put a 180 days option in the history candle stick graphs (EvE has it).
4) An improvement for your history graphs: the ability to specify a start (or end) date for the first (or last) bars / dots. So one can ie. analyze 90 days of the market at the end of 2011 instead of being forced to look at today.
Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
6ie
The Kiwis
3
|
Posted - 2012.02.14 02:07:00 -
[449] - Quote
"Mega Af".....wait..... drops down 2 options.....click..... BAM! Right to the items page. You Rock :) Very nice fix on the searching! Thanks a lot. |
Aineko Macx
Royal Amarr Institute Amarr Empire
139
|
Posted - 2012.02.14 06:13:00 -
[450] - Quote
Callean Drevus wrote:I need specifics on the items where it's going wrong though, that will help with my investigation. Can you provide me with region/type combinations? For instance, the buyHighest5 here: http://www.evemarketeer.com/api/info/26088/csv/10000002
Quote:I shouldn't be, it should even be a little less, as I'm not interested in tech 2 BPO's. Where are you getting this number? Here: http://www.evemarketeer.com/api/region/csv/10000002
I track this subset myself, which are all available market items minus BPs, skills, reactions and supercapitals:
SELECT typeID, g.groupID, g.categoryID, typeName FROM invTypes t, invGroups g WHERE marketGroupID IS NOT NULL AND t.groupID = g.groupID AND t.published = 1 AND g.categoryID NOT IN(9, 16, 24) AND g.groupID NOT IN (30, 659) |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
100
|
Posted - 2012.02.14 06:56:00 -
[451] - Quote
Vaerah:
2) Price filtering is not optional now at this moment (though if it ain't working either I guess it makes sense I don't get complains :P), but I've long been planning to just mark the suspicious orders, instead of delete them.
3) Good thing about low prio stuff is that it generally only takes a few seconds to do it :)
4) you can still select 720 or 1800 and use the zooming feature, but I agree that setting dates might be more intuitive. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
271
|
Posted - 2012.02.14 07:47:00 -
[452] - Quote
Callean Drevus wrote: 4) you can still select 720 or 1800 and use the zooming feature, but I agree that setting dates might be more intuitive.
I noticed that under "trends" we get a nice black empty area.
I would fill it with some thingies:
1) A legend stating "drag to zoom", because it's not exactly obvious that the feature exists.
2) (A button to zoom in)
3) A button to zoom out once you used 1)
4) A button to reset to "original" scale, the one the graph appears when you first see the page. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Maeste Madeveda
The Spawning Pool Team Liquid
2
|
Posted - 2012.02.14 09:11:00 -
[453] - Quote
I have tested the T2 reprocessing and it seems not to beat the benefits the reprocesser finder finds. Any info on the calculation method ? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
100
|
Posted - 2012.02.14 19:00:00 -
[454] - Quote
Maeste Madeveda wrote:I have tested the T2 reprocessing and it seems not to beat the benefits the reprocesser finder finds. Any info on the calculation method ?
What exactly did you try to say? Is this good or bad? I noticed that my new calculation method that makes use of the real trade value of items has some flaws :P since people really are stupid enough to buy WAY above price :P Developer/Creator of EVE Marketeer
|
Maeste Madeveda
The Spawning Pool Team Liquid
2
|
Posted - 2012.02.15 00:41:00 -
[455] - Quote
Uhm magnificent typo. I wanted to write bear, and I wrote beat...
Anyway it was before the new changes so you might fixd the problems. I reprocessed a cheap T2 to know what I cold get back from it and I discovered that you don't get T2 components but mostly minerals. That would be okay if the calculations done by the site were accurate but they were not. So I was wondering how you calculate the reprocessed materials : do you use our skills ? do you know which components are given back ? etc |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
100
|
Posted - 2012.02.15 09:47:00 -
[456] - Quote
As far as I know, when reprocessing, the base materials, minus a percentage are returned. I've never actually tried reprocessing a carrier to see whether I could get back the capital construction components though. But this is the premise that reprocessing profits are based on.
The new calculation method takes this into account when determining the prices of the items. It also doesn't count lower than one level, which was messing up the previous calculation (if there were things in the extra materials list that were comprised of base materials, it would still count them as base materials you'd get back for reprocessing, which wouldn't happen of course).
The only problem I'm aware of regarding reprocessing stuff now, is that some prices are unrealistically high, but the idiotic part is that I'm now using the average price of the last 30 days (according to history data), which should be pretty accurate. Developer/Creator of EVE Marketeer
|
Desmont McCallock
127
|
Posted - 2012.02.15 10:26:00 -
[457] - Quote
The data for a reprocessed item can be retrieved from SDE invTypeMaterials table (as Callean must already know). Indeed a reprocessed carrier returns capital construction components as do all capital ships. |
Vaerah Vahrokha
Vahrokh Consulting
275
|
Posted - 2012.02.15 22:24:00 -
[458] - Quote
Today I tried creating my own upload group.
I entered a new group name in the group name edit box (duh!) I then entered one item name and I got a nice pull down with the grep of the item name.
I proceeded for more items till I was satisfied.
But now how do I use this list? I find no buttons to save it nor to upload it (using history price uploads). Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Maeste Madeveda
The Spawning Pool Team Liquid
2
|
Posted - 2012.02.15 23:12:00 -
[459] - Quote
I second what Vaerah said. Plus I haven't found any way to delete already written lists.
On the subject of processing, you might want to check why the reprocessing values of JF are over 100B. I guess it's because of the prices of capital parts that a dumb someone is putting on the market.
Other than that, you are doing a great job. Have some encouragement.
YAY for Callean
_o/ \o/ \o_ |
Aineko Macx
Royal Amarr Institute Amarr Empire
139
|
Posted - 2012.02.16 06:39:00 -
[460] - Quote
Another idea for filtering the highest buy / lowest sell: Don't select 5% of the available or demanded volume, but rather of the daily traded volume.
Seconding YAY for Callean |
|
Steve Ronuken
Fuzzwork Enterprises
232
|
Posted - 2012.02.16 19:05:00 -
[461] - Quote
Anyone else have problems with the unified uploader not picking anything up, after the patch today? (I've restarted the uploader.) FuzzWork Enterprises http://www.fuzzwork.co.uk/ Blueprint calculator and other 'useful' utilities. |
Scrapyard Bob
EVE University Ivy League
692
|
Posted - 2012.02.16 19:38:00 -
[462] - Quote
Steve Ronuken wrote:Anyone else have problems with the unified uploader not picking anything up, after the patch today? (I've restarted the uploader.)
Only if you don't shut down the uploader before patching. Every time the EVE client patches, it moves the old cache directory out of the way and creates a new one.
Also, if you run multiple clients off of multiple junction points, be sure to patch all junction points by starting up a client at least once on each junction point to get the client update for that junction. |
Scrapyard Bob
EVE University Ivy League
694
|
Posted - 2012.02.17 16:54:00 -
[463] - Quote
Various wish lists:
1) Large POS tower fuel cost per 30-days graph. Take the input costs for the (4) fuel block BPOs, assume ME 40, assume Production Efficiency IV, then graph it out based on the historical trade volume from the 4 market hubs. Large towers consume 1 batch (40 blocks) of fuel per hour.
2) Ore value per m3 graph. Show how ISK/m3 has varied over the past few months/years and which ores are at the top of the ISK/m3 pile. Use the refined mineral costs (assuming zero-waste) and base mineral costs off of the hi-sec market hub trade histories.
3) Ice value per block graph. Show the refined value and/or show the block trading price for the different ice types. You may want to divide it up so that there is a hi-sec chart (only the hi-sec ices) and a null/low chart with the fancier ices (it's harder to get good trade values for the null-sec ice).
4) Total traded ISK volume per market hub / system / region over time for the top (8) stations/systems/regions.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
100
|
Posted - 2012.02.17 22:57:00 -
[464] - Quote
Damn, damn, I knew this would happen. I'm gone for a few days, and people are cheering me on and requesting even MORE features :P
That said, I've been away for a while because I have been busy fixing my companies production server, which broke down for one reason or another, that I still haven't been told. All the while, the EMK server was running fine, which made me feel exceptionally stupid. I'd rather have you all angry than my clients, but **** happens I guess :P
As a side effect, both my new production server (at the same place where EMK is hosted) and the EMK server are now costing me about 10 dollars a month more, but also fully back upped every day.
Quote:Only if you don't shut down the uploader before patching. Every time the EVE client patches, it moves the old cache directory out of the way and creates a new one. I suddenly understand why EMD has this HUGE warning about this that pops up every time you open the uploader :)
Oh, and will log all feature requests, but probably not work on them for now, as I'm still firefighting
Quote:I second what Vaerah said. Plus I haven't found any way to delete already written lists.
On the subject of processing, you might want to check why the reprocessing values of JF are over 100B. I guess it's because of the prices of capital parts that a dumb someone is putting on the market. Ehrm, yeah, the custom list mechanism is quite old, and the interface has changed a bit, plus, I haven't added all the buttons I probably should've added.
To create a custom list, you first enter a group... now wait, I just checked, and I broke it completely when rewriting the uploader interface. I didn't even modify any part of it, but somehow it is broken now. It'll give me an opportunity to make it somewhat nicer that it is now.
Quote:4) Total traded ISK volume per market hub / system / region over time for the top (8) stations/systems/regions. I do not think it is possible to do any more than checking this for a region, but it will still be interesting. I had the power to do it for individual orders once (thus also enabling hub/system stats) but I've disabled recording every trade a long time ago. May start it again some time though (still have the function more or less built in). Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
697
|
Posted - 2012.02.18 02:39:00 -
[465] - Quote
Callean Drevus wrote:Quote:4) Total traded ISK volume per market hub / system / region over time for the top (8) stations/systems/regions. I do not think it is possible to do any more than checking this for a region, but it will still be interesting. I had the power to do it for individual orders once (thus also enabling hub/system stats) but I've disabled recording every trade a long time ago. May start it again some time though (still have the function more or less built in).
I envision you just summing up all of the market history information for each region into a "total ISK traded per day" table (primary key: region + date). Then graph the total ISK/day for each of those regions. Offer a 5-day moving average and 20-day moving average variant to smooth out bumps and to more clearly show the rise-and-fall of the various trade regions.
Whether you hand calculate which (4 / 8 / 10) regions to show on the graph would be up to you. |
Snarf Aldes
University of Caille Gallente Federation
0
|
Posted - 2012.02.18 14:43:00 -
[466] - Quote
Both on the order info on the site, and the info provided with the api, when i look at Sterile Conduits (id = 2875), i don't see any buy orders with a minimum quantity greater than 1. There are at least 2 buy orders atm. that don't show up (one with 200 and one with 1000 min. qty.) in Jita 4-4
Is it a bug or a feature?
thnx.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
101
|
Posted - 2012.02.18 17:25:00 -
[467] - Quote
Snarf Aldes wrote:Both on the order info on the site, and the info provided by the api, when i look at Sterile Conduits (id = 2875), i don't see any buy orders with a minimum quantity greater than 1. There are at least 2 buy orders in in Jita 4-4 at the moment that don't show up (one with 200 and one with 1000 min. qty.)
Is it a bug or a feature?
thnx.
This is a feature. Orders which have a minimum quantity higher than half of the total quantity are filtered, because they are likely scams. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
288
|
Posted - 2012.02.19 13:40:00 -
[468] - Quote
Some more things:
Since I cannot create my own upload group (yet) I thought to just click on the relevant items in history mode, start the uploader and have it upload them.
- It did not work, the item ID never appeared in the uploads.
- I went to check for the logs, I can find plenty of error dump logs but no log about the last upload (the stuff you see scrolling in the status tab. Is there a way to have it saved?
- I copy pasted the status window text and then searched for the item ID above but still nothing.
How do I force the uploader to upload history I have for sure? My own cache scraping script based on the same engine gets the updated history. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
101
|
Posted - 2012.02.19 14:30:00 -
[469] - Quote
This seems like an issue with updating and keeping the uploader running while doing so. In that case it might not detect the items you are uploading. Otherwise, I have no idea why it is negating your manual items.
Just to make sure, you have selected the history tab in the market view (that is, not the website page, but the ingame market browser) have you? Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
289
|
Posted - 2012.02.19 16:37:00 -
[470] - Quote
Callean Drevus wrote:This seems like an issue with updating and keeping the uploader running while doing so. In that case it might not detect the items you are uploading. Otherwise, I have no idea why it is negating your manual items.
Just to make sure, you have selected the history tab in the market view (that is, not the website page, but the ingame market browser) have you?
I could not use manual items, because of the issue I posted some days ago: once created the list I found no way to save it or make it execute.
But since I noticed the uploader sent stuff even with no game running (thus no in game browser open), I tried starting it alone with no game. It uploaded some things but not the items I wanted. They were surely in the cache because I clicked on yearly history in game on each of those items and my scraping script sees these cached items updated. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
102
|
Posted - 2012.02.19 16:59:00 -
[471] - Quote
Is it possible your own caching script is deleting files right after they are read? Or locking the files and not freeing them again?
Basically, if you do this:
1. Start uploader 2. Start game 3. Click on item in market window (history or orders)
The uploader should upload. If it isn't doing something along that path, something is wrong.
Version 0.5 (fixed in 0.6) read all already existing cache files on starting (the uploads you saw) and tried to upload those, but 0.6 ignores them, because it led to duplicate uploads. Developer/Creator of EVE Marketeer
|
Maeste Madeveda
The Spawning Pool Team Liquid
2
|
Posted - 2012.02.19 20:30:00 -
[472] - Quote
The new 0.6 version seems to run fine on my computer. |
Vaerah Vahrokha
Vahrokh Consulting
290
|
Posted - 2012.02.20 03:05:00 -
[473] - Quote
Callean Drevus wrote:Is it possible your own caching script is deleting files right after they are read? Or locking the files and not freeing them again?
Basically, if you do this:
1. Start uploader 2. Start game 3. Click on item in market window (history or orders)
The uploader should upload. If it isn't doing something along that path, something is wrong.
Version 0.5 (fixed in 0.6) read all already existing cache files on starting (the uploads you saw) and tried to upload those, but 0.6 ignores them, because it led to duplicate uploads.
My script does not delete any cache file and I only started it *after* I noticed the lack of updates. I'll keep you informed about how it goes with the new uploader. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Vaerah Vahrokha
Vahrokh Consulting
290
|
Posted - 2012.02.20 03:14:00 -
[474] - Quote
Callean, you know we have brainstormed the OHLC graph and export some time ago.
Now it's time to see the fruits.
I have posted a brief intro and tutorial about your web site. It shows how to prepare OHLC data good to be imported on a RL trading platform.
The article is: EvE Marketeer GÇô powerful EvE trading oriented website
There is also a step by step tutorial on how to import the generated files into a standard RL trading platform (so you can physically see the results of your efforts): How to import EvE market data in a trading platform
Finally, there's an actual market analysis and trade example available here: EvE Online GÇô Nitrogen Isotopes GÇô 2012-01-18 GÇô A very profitable trade
A 10B investment resulted in about 2.4B net profit.
This can also be done with my script (almost identical output) but your website makes it way much easier and quicker. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Vaerah Vahrokha
Vahrokh Consulting
294
|
Posted - 2012.02.20 16:07:00 -
[475] - Quote
Issue with uploader 6: it uploaded data once (345 files) then stopped. Now regardless whether I start before or after EvE, nothing gets sent. I have (and always will have) the "delete cache files" option disabled.
Here's a screenshot of history market (I did select "history mode" in the IGB browser) running with the uploader at the top. As you see no log nothing despite the items I am uploading were last updated yesterday.
If I go in "recent uploads" I see nothing have been uploaded. It worked with uploader 5 (with some issues I posted in the past). I put the upload key in the Endpoints as well.
All I get in the log files is "Logger started" repeated some times. The auto detected cache location is correct. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Snarf Aldes
University of Caille Gallente Federation
0
|
Posted - 2012.02.20 18:01:00 -
[476] - Quote
Callean Drevus wrote: For all those of you whom have their own publically accessible endpoints, you can now add these to the EMK website, to allow people to add these preconfigured to the Unified Uploader.
Cool addition, works like a charm.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
103
|
Posted - 2012.02.20 20:41:00 -
[477] - Quote
Vaerah Vahrokha wrote:As you see no log nothing despite the items I am uploading were last updated yesterday.
If I go in "recent uploads" I see nothing have been uploaded. It worked with uploader 5 (with some issues I posted in the past). I put the upload key in the Endpoints as well.
Starting from version 0.6, it will not upload anything that is already in the cache when the uploader is started. In addition, the default EVE problem of only updating the item only once every time you start it unless something changes (for history, probably only once a day), probably applies.
Can you tell me whether the steps I outlined above work? You should follow those while you have no application (well, not EVE or the uploader anyway) open.
In addition, you should really delete some of the cache files eventually (or move them.. I forgot to add that option...) or your EVE market seems to get really slow.
I may have to write an essay about the intricacies of those damn cache files :P Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
301
|
Posted - 2012.02.20 23:42:00 -
[478] - Quote
Callean Drevus wrote:As you see no log nothing despite the items I am uploading were last updated yesterday.
Can you tell me whether the steps I outlined above work? You should follow those while you have no application (well, not EVE or the uploader anyway) open.
Tried the steps above very carefully from rebooted computer, no upload starts, the numbers stays at 345 files read like this morning. Switched between Market data and history tab, nothing happens.
It just says "Welcome to the new EVE Marketeer uploader" and that's it.
I clicked on an item I never clicked before and THEN that item got uploaded.
Then I started the IGB and loaded the upload page. I selected the minerals, I have not played since before DT, so something HAS to have changed in the history (or not?).
But nope.
I then selected the market orders (only minerals), then pressed the IGB play button and this time it uploaded the stuff. Notice how it uploaded the orders with the "orders in past 24 hours" pane saying 100%.
It does not update the history despite the "History in past 7 days" is 91% (i.e. data is missing)
So, data is missing but not sent? I am sending 1 year of history. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Vaerah Vahrokha
Vahrokh Consulting
302
|
Posted - 2012.02.21 16:09:00 -
[479] - Quote
I ran the uploader again. Today it sent the history up.
This opens to 2 considerations:
1) It won't send current day data (which is ever changing, therefore would always trigger a send).
2) History mode basically is a one run now. Therefore the play button should only run through the selected category (i.e. minerals, PI...) from the first item to the last item once. Once at the end of the list, it should not restart from the beginning of the list like it is doing now and restart the cycle forever. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
103
|
Posted - 2012.02.21 18:28:00 -
[480] - Quote
Quote:1) It won't send current day data (which is ever changing, therefore would always trigger a send).
2) History mode basically is a one run now. Therefore the play button should only run through the selected category (i.e. minerals, PI...) from the first item to the last item once. Once at the end of the list, it should not restart from the beginning of the list like it is doing now and restart the cycle forever. This seems a plausible explanation. Making the history not cycle is not really a solution though, in my opinion. People will generally leave the uploader running when they aren't doing anything else, so stopping after one cycle wouldn't make much difference, and just annoy people whom would want to keep uploading all the time (though downtime would eventually kill that anyway, I admit).
Still, for others, it might be a good idea to just make a checkbox or separate start button that will start just one cycle of the uploads.
Quote:Please add the ability to permalink the graph panes. Yes! This has been in my mind for a long time, but for some reason I constantly forget or postpone doing it...
Quote:As you can see I am heavily linking and sponsoring your website everywhere both on the EvE forums and on my website, hope you like it. I like it, but it won't get you preferential support well, maybe a bit, since bad publicity is not the kind I really want :P Developer/Creator of EVE Marketeer
|
|
Maeste Madeveda
The Spawning Pool Team Liquid
3
|
Posted - 2012.02.21 20:59:00 -
[481] - Quote
I also encountered a problem today with the uploader, it was stuck at some point but I was unable to reproduce the error because it went well after I reloaded the app.
Edit : The new compare system is very useful, but you have to remove Lonetrek and replace it by Heimatar. There is no real hub in Lonetrek (at least no complete ones) and Rens is certainly more important than Lonetrek. |
Scrapyard Bob
EVE University Ivy League
707
|
Posted - 2012.02.22 22:49:00 -
[482] - Quote
The API pages have gone fugly. With the new backgrounds for Parameters / Log Format / etc, it's impossible to read the text without selecting it.
Also, it would be nice if we could link directly to specific API explanation pages instead of saying "go to this URL, then scroll down a lot and look for XYZ".
|
TheSmokingHertog
Black Hole INC
18
|
Posted - 2012.02.22 23:00:00 -
[483] - Quote
Could we get an option to have all HighSec / LowSec / NullSec prices listed within the product overview?
Atm I use EVEcentral for that, but it would be great to have this all in 1 place. Good thing to note: I would prefer a filter to, I mostly wholesale, so I would like to filter the orders on a case by case bases. Say > 100000# or >1000#, due the product type, atm this seems not possible at @ evemarketeer. |
Vaerah Vahrokha
Vahrokh Consulting
310
|
Posted - 2012.02.24 10:04:00 -
[484] - Quote
2 little bugs.
1) In candle bars graph, there's a nice mouse over tooltip that tells open, close etc. But its Z-Order makes it go below the selectors for the months and history.
Screenshot
2) Also, in the same screenshot, you can see how sometimes the candle bars appear with no body at all. It should always show at least 1 pixel. The item in the screenshot where you can see this behavior is Single-crystal Superalloy I-beam Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Amber Green Thorn
Federal Navy Academy Gallente Federation
45
|
Posted - 2012.02.24 18:56:00 -
[485] - Quote
Superb stuff - bit difficult I could imagine for a new player to get his head around but whoa -nice one! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
107
|
Posted - 2012.02.25 21:19:00 -
[486] - Quote
I am aware of the problems with some of the parts of the interface, however, I'm super busy in RL right now, so updates will have to be a bit delayed.
Regarding the comparisons, those can be done changed on the same page by using the plus button, and can be removed on the group summary page (click one of the links in the breadcrumbs to get there) . A link to delete on the same page will follow.
I will also mention, that though I would very much like a item summary page for combined regions, the architecture makes this a bit difficult. Would it suffice if the page would just serve the item info table and the orders list (no trends, history, manufacturing etc. At least at first?)
Good idea on splitting up the API pages, would probably make more sense and give me less headaches. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
317
|
Posted - 2012.02.26 20:29:00 -
[487] - Quote
I keep having problems with the uploader (I don't delete cache files).
If I click on an item I never uploaded it works. If I click on an item I uploaded it won't work, neither in manual nor IGB mode and yes I repead the procedure you listed some posts ago.
Today I tried uploading all minerals (I did not do it since days) and it uploaded nothing at all, neither the orders nor the history.
BTW please add a category "Ice" (at least isotopes, but LO etc would be cool) so I can use the IGB for them.
Result: saving CSV extracts a file with prices of 2 days ago. My own script run over the same cache fetches all the prices, proving I indeed have the full data on my computer. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
TheSmokingHertog
Black Hole INC
18
|
Posted - 2012.02.27 11:42:00 -
[488] - Quote
Callean Drevus wrote:...
I will also mention, that though I would very much like a item summary page for combined regions, the architecture makes this a bit difficult. Would it suffice if the page would just serve the item info table and the orders list (no trends, history, manufacturing etc. At least at first?)
....
As far as my function request is concerned, I would not need trends to go with the combination of regions. The trends, history and manufacturing would be available through means now available. Within product groups I have on the markets the named tabs would not have additional value, since I have enough tools available to notice inter-regional market trends at this moment.
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
108
|
Posted - 2012.02.28 21:10:00 -
[489] - Quote
Vaerah Vahrokha wrote:I keep having problems with the uploader (I don't delete cache files).
If I click on an item I never uploaded it works. If I click on an item I uploaded it won't work, neither in manual nor IGB mode and yes I repead the procedure you listed some posts ago.
Today I tried uploading all minerals (I did not do it since days) and it uploaded nothing at all, neither the orders nor the history.
BTW please add a category "Ice" (at least isotopes, but LO etc would be cool) so I can use the IGB for them.
Result: saving CSV extracts a file with prices of 2 days ago. My own script run over the same cache fetches all the prices, proving I indeed have the full data on my computer.
I think I now know what the problem is. The uploader only checks for filenames. Because your files are always kept, and are only overwritten (with the same name) they uploader doesn't detect changes, and therefore uploads nothing. I'll modify the check to take into account file creation date as well :)
Oh, in addition to simply creating a 'move cache instead of delete', since that will solve these kinds of problems forever. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
332
|
Posted - 2012.02.28 21:48:00 -
[490] - Quote
Very cool, Callean.
I'd love the "file date-time" check because my other programs all auto-detect EvE cache file location and moving them will make those softwares stop working. It will also make the whole EvE caching mechanism pointless.
Second thing:
The search edit box is too short, items like Single-crystal Superalloy I-beam get cut very short. There's some room to the right of the search box or its buttons could be put outside of it.
Edit: bug report.
I was wondering about mismatches between the in game values and the graphs till I noticed this:
1) I opened Robotics
2) The resulting page is in this screenshot.
Notice how it states:
"Updated 2012-02-28"
while the last candle bar is off Feb 25.
Now I am going to try upload a newer data but this is just one of the items affected by this same thing. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
108
|
Posted - 2012.02.28 23:09:00 -
[491] - Quote
The EVE caching mechanism is pointless anyway, since EVE actually becomes faster when you clear the cache (the market window, anyway). At least, this was so in older versions of EVE. I haven't bothered to try out filling my cache lately.
While I agree that some items do not fit in the search box. I do not think this happens very often (to the point where you do not know what you typed into it anymore anyway). If I make the box bigger, or more the items out of it, the whole bar will be filled, and that will be very ugly (when you are logged in, you have some more options).
What I can do is expanding it on focus though, so you will be able to see all of what you type even if it disappears afterwards.
Regarding the last issue you mentioned. The value 'last updated' corresponds to is the order information. The last update for history information is not currently shown.
UPDATE:
I added some other things tonight (since I won't be able to add anything tomorrow).
1. ISK Dollar Index (found under trade information) 2. Likely sell locations for items (found on the item summary page, a new tab), that allow you to see where the items are currently sold (in the selected region) and where it could be profitable to sell them, based on asteroid belts or missioneers. 3. The comparison between regions (on the item summary page) remembers what you last selected, and now allows you to select none, if you think it is an annoying feature that just takes up screen space Developer/Creator of EVE Marketeer
|
Maeste Madeveda
The Spawning Pool Team Liquid
3
|
Posted - 2012.02.28 23:57:00 -
[492] - Quote
In the LP store, it would be nice if we could have the blueprints too. I know the prices of blueprints is not realistic because people are going to manufacture it asap but it could be nice to have them featured. Also there is a SQL error at the first page of the Best LP page :
Quote:MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ORDER BY corporationName' at line 1 MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'GROUP BY si.item_id ORDER BY profit_per_lp DESC' at line 1
Aside from that, I'm toying with the new features, it looks great. Sometimes it seems not to work but I can't figure why yet so I'll get back to you when I do. |
Vaerah Vahrokha
Vahrokh Consulting
334
|
Posted - 2012.02.29 08:13:00 -
[493] - Quote
Callean Drevus wrote: Regarding the last issue you mentioned. The value 'last updated' corresponds to is the order information. The last update for history information is not currently shown.
It'd be lovely if that last history update was shown, for several reasons:
1) The devices used to read the web page could have no easy way to hover over the graph on the last day. The graph for obvious space reason won't show all the days below the graph.
2) It's stated nowhere that the last update is only for the orders book, if I can get fooled stay sure that others will as well.
3) Knowing at a glance that the history is not updated would make me turn on my EvE computer and upload it. I only work with the history anyway.
Callean Drevus wrote: 1. ISK Dollar Index (found under trade information)
I love you!
On an unrelated matter, how the hell do you manage to code in all those cool looking and useful things so fast? Last time I PHP coded it was a big effort (time wise) and easily produced ugly pages. Are you using a framework like YII or just divine powers? Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
110
|
Posted - 2012.02.29 10:17:00 -
[494] - Quote
Quote:On an unrelated matter, how the hell do you manage to code in all those cool looking and useful things so fast? Most of all, I think it's that I hack together the stuff in a framework I built myself, it isn't really the best form of architectural coding (that is, I could spend twice as much time, and end up with somewhat prettier code), but it suffices. It is easy to add things that fall within the framework, but an insane expenditure of time to fix things that fall outside of that fold (for example, combining multiple regions).
In addition, the ISK Dollar Index is just the other index pages with a different and simpler query, so that one was exceptionally easy
Quote:Also there is a SQL error at the first page of the Best LP page : Fixed it
I think the prices of blueprints are off the charts because there are no prices at all. This wasn't planned like this (wanted to show everything), I'll see if I can add them. Developer/Creator of EVE Marketeer
|
Jaqa
Screaming War Eagles Incorporated
2
|
Posted - 2012.03.03 08:15:00 -
[495] - Quote
Bumping. With love. The Original Jaqalope Pilot of the Transportosaurus Rex 100% Free Range |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
111
|
Posted - 2012.03.03 17:35:00 -
[496] - Quote
Today, added a (if I may say so myself) really cool feature.
Custom item lists! You hear this correctly, you can now create lists of item types you want information about. Currently, there is not much use for that though, as there's only one page that can make use of it. Anticlimax?
That page however, is really rather cool. It is also the most complex query I've ever written for EMK. With the possible exception of the traderoute finding alghorithm (which has never been released, yet).
In any case, I had always known it was a good idea to just buy stuff in Jita, and sell it in other solar systems for a markup, because people are lazy. However, I always had problems identifying the systems where it would be a good idea to sell stuff. Partially due to now know what missioning systems were, or what agents were available, but also because checking whether there already were quite enough competitors would prove to be a days worth of work.
As it turns out, this has also proven to be a days worth of work, but now I never have to do all that again. This report will check the region for all the items you have added to the list, and give you an overview of all solarsystems in which those are sold, with some additional information to make the spotting of lucrative areas quite easy.
The easiest way to explain is to just have you try it out. Just be careful, if you enter a lot of item types, the report might take a while to run (that is, 25 is probably the max if you want it to finish anywhere in a normal span of time, if you don't mind waiting, feel free to add more items :P)
Developer/Creator of EVE Marketeer
|
Dieter Rams
State War Academy Caldari State
9
|
Posted - 2012.03.03 20:29:00 -
[497] - Quote
Very cool, thank you! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
111
|
Posted - 2012.03.03 20:32:00 -
[498] - Quote
****, it makes sense I should tell you that doesn't it
Account -> Lists Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
367
|
Posted - 2012.03.04 09:01:00 -
[499] - Quote
A little new suggestion.
For obvious reasons, no trading platform (nor your graphs) show all the days on the time axis.
All the platforms show a nice tooltip as you do already, with the minimum, maximum etc, but they also add the date of the price candle / dot as first line of the tooltip itself. You should too imo, it makes muuuuch sense and it takes like 30 seconds to implement. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
111
|
Posted - 2012.03.04 11:51:00 -
[500] - Quote
It actually doesn't take 30 seconds, because the graphing lib is so counterintuitive :P but yes, I'll see if I can do it Developer/Creator of EVE Marketeer
|
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
111
|
Posted - 2012.03.05 19:50:00 -
[501] - Quote
Just wanted to let you guys know I'm currently negotiating with Covert Kitty to see whether I'll buy her traderoute finding tool.
Any donations towards the 5B cost are appreciated (though not strictly necessary) . Developer/Creator of EVE Marketeer
|
Dieter Rams
State War Academy Caldari State
9
|
Posted - 2012.03.06 01:04:00 -
[502] - Quote
I added a manual group and a manual item to market scanner and now I can't delete them, I get this:
Quote:Warning: implode(): Invalid arguments passed in /var/www/www.evemarketeer.com/mod/uploader.php on line 77 MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1
Also one of my groups is named ($type). |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
113
|
Posted - 2012.03.06 17:04:00 -
[503] - Quote
Ah yes, manual groups are kind of totally broken. I'll implement the same item list system that I'm currently using for reports into the manual item selection screen before the week is done. In the meantime, you can already create item lists when you go to 'Account -> Lists'. Developer/Creator of EVE Marketeer
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
113
|
Posted - 2012.03.12 20:45:00 -
[504] - Quote
I apologize for my unableness to fix manual items. And apologize more for not being able to do so in the coming time.
I have also added a poll to the EVE Marketeer homepage to gouge interest in having trade finding software (reportedly the best) bought. Usage will be free of course. Developer/Creator of EVE Marketeer
|
Shellac Brookdale
RAZOR Alliance
4
|
Posted - 2012.03.13 19:15:00 -
[505] - Quote
The unified uploader doesnt work for me. I've found the following error message in the log file
Traceback (most recent call last) File "emkUploader.py", line 19, in File "wx\_core.pyo", line 7981, in __init_ File "wx\_core.pyo", line 7555, in _BootstrapAp File "emkUploader.py", line 11, in OnIni File "UploaderFrame.pyo", line 117, in __init_ File "UploaderFrame.pyo", line 178, in initializ File "UploaderFrame.pyo", line 430, in installationAutodetectButtonClic File "os.pyo", line 423, in __getitem_ KeyError: 'LOCALAPPDATA
Theres no such environment variable in my windows XP version. Even after creating it, I really couldn't get it to work. My EVE installation is located at g:/games/EVE and to uploaded expects a CCP folder. Why not just making the path configurable through the config.ini? |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
114
|
Posted - 2012.03.13 20:19:00 -
[506] - Quote
I think you can actually edit it trough the config.ini? If you enter it there, I believe it will not automatically try to look up your installations anymore.
It should look like this:
[installations] installation_count = 1 installation_0 = C:\Users\Bart\AppData\Local\CCP\EVE\c_program_files_(x86)_ccp_eve_tranquility
That said, I'll see if I cannot make it optional, since it could potentially be annoying if issues like this exist. Developer/Creator of EVE Marketeer
|
Shellac Brookdale
RAZOR Alliance
4
|
Posted - 2012.03.13 20:53:00 -
[507] - Quote
My installations section currently looks like this:
[installations] installation_count = 0
I'll try to add a installation_0 value and see if I can get it to work.
|
Scrapyard Bob
EVE University Ivy League
749
|
Posted - 2012.03.14 17:55:00 -
[508] - Quote
On the regional prices tab, I'd like to see (2) filters up top.
1) Only show regions with updated prices within the last 6 / 12 / 24 / 48 / 72 hours
2) Only show null or only show empire regions
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
114
|
Posted - 2012.03.14 19:46:00 -
[509] - Quote
Shellac Brookdale wrote:My installations section currently looks like this:
[installations] installation_count = 0
I'll try to add a installation_0 value and see if I can get it to work.
Please do not forget to turn installation count to 1 then (I assume you were going to do this, but just to be certain )
@Scrapyard: Recorded Developer/Creator of EVE Marketeer
|
Shellac Brookdale
RAZOR Alliance
4
|
Posted - 2012.03.18 11:56:00 -
[510] - Quote
I've added my installation path manually to config.ini and the client will start now. However, nothing happens. All upload stats will be set to zero. Browsing the market doesnt do anything in the client. Logs show nothing.
|
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
114
|
Posted - 2012.03.18 12:31:00 -
[511] - Quote
Shellac Brookdale wrote:I've added my installation path manually to config.ini and the client will start now. However, nothing happens. All upload stats will be set to zero. Browsing the market doesnt do anything in the client. Logs show nothing.
Be aware you have to add your 'eve application data' path to the config, not the installation directory of EVE itself.
In my install, this is C:\Users\Bart\AppData\Local\CCP\EVE\c_program_files_(x86)_ccp_eve_tranquility. This folder contains two other folders, called 'cache' and 'settings'.
Not exactly sure whether this would be on your computer. I believe in XP it must be C:/Documents and Settings/Bart/Application Data or something, application data is generally a hidden folder, so you might have to turn show hidden folders on to actually see it.
Developer/Creator of EVE Marketeer
|
Shellac Brookdale
RAZOR Alliance
4
|
Posted - 2012.03.18 14:49:00 -
[512] - Quote
Found the directory and got it working now. Thanks!
Btw. how many API calls do you allow per minute/hour/day? Can you give me any number, so I can be safe you won't block my application at some point?
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
114
|
Posted - 2012.03.18 15:05:00 -
[513] - Quote
Shellac Brookdale wrote:Found the directory and got it working now. Thanks!
Btw. how many API calls do you allow per minute/hour/day? Can you give me any number, so I can be safe you won't block my application at some point?
I just trust that if you do too many calls, everything will become very slow, and you'll stop doing as many calls. If anything truly becomes slow, I'll investigate and block someone, but that hasn't ever happened yet. Developer/Creator of EVE Marketeer
|
Sokor Loro
Dreddit Test Alliance Please Ignore
34
|
Posted - 2012.03.20 03:25:00 -
[514] - Quote
Love your site. I've been using your API to pull data to build some spreadsheets for PI.
I am trying to look at what to produce by looking at the amount of items are up for sale in Jita and how many of them have been bought, and creating some sort of index for it.
So a question or two:
What is demand? You give it as, "demand: the total amount of items bought". Is this in the past day? 90 days?
Secondly, I'm trying to create a good indicator of what to produce based on your API pulls for sell_lowest5, supply_in_5, and demand. Basically trying to look at how much is currently on the market, how much is being bought from the market, and at what prices. Care to give any advice/guidance on this?
Both of these are open to anyone, could really use the help! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
114
|
Posted - 2012.03.20 20:28:00 -
[515] - Quote
Ah, demand is as, the total quantity of all buy orders together, at this point in time. To see the amount of items bought, you should try looking at movement, and the estimated buy/sell. This is an average per day.
As to a good indicator of what to produce, I have no idea, I'm only providing the data ^^ but there are probably a lot of people whom can tell you how to do this. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
782
|
Posted - 2012.03.26 03:06:00 -
[516] - Quote
Uploads are not being processed, about an 89k backog at http://www.evemarketeer.com/statistics/uploads |
Domi Etymology
Violent Alternatives C0NVICTED
0
|
Posted - 2012.03.28 12:25:00 -
[517] - Quote
lol 152k now. Time to crank up every account to upload, see if we can't get this ***** over 1 mil and give Callean one helluva headache when he notices :) |
Vaerah Vahrokha
Vahrokh Consulting
454
|
Posted - 2012.03.29 18:04:00 -
[518] - Quote
First of all, kudos for having put anchors to be able and link the item details tabs!
Suggestion: add the ability to specify the time frame (i.e. weekly bars) and days resolution (i.e. 360) as anchors.
Possible bug: I think I found a bug: weekly candles (montly too maybe) seem to be drawn starting from today. I.e a weekly candle would show from last Thursday to this Thursday or similar. The correct calculation is to draw from the beginning of the nth previous session to nth-1 session. I.e. Today's candle would be a candle formed by Monday to today candle, the previous candle would be from 2 mondays ago to last Sunday and so on. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Vaerah Vahrokha
Vahrokh Consulting
456
|
Posted - 2012.03.30 10:02:00 -
[519] - Quote
Yes the website uploads died. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Scrapyard Bob
EVE University Ivy League
803
|
Posted - 2012.03.30 13:53:00 -
[520] - Quote
Market Scanner page has gotten very unreliable over the past week or two. Sometimes when you hit the "play" button, it does the countdown and starts changing the item displayed, but never opens up the market window in-game.
(I've had to switch to using EMD or EC sites to do the scanning.) |
|
Vaerah Vahrokha
Vahrokh Consulting
459
|
Posted - 2012.03.30 23:21:00 -
[521] - Quote
More than 500k uploads backlog, please unlock it. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Scrapyard Bob
EVE University Ivy League
807
|
Posted - 2012.03.30 23:41:00 -
[522] - Quote
Fix it fix it fix it? http://www.youtube.com/watch?v=1Isjgc0oX0s
540k backog now. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
116
|
Posted - 2012.04.01 10:24:00 -
[523] - Quote
Sorry guys, after fanfest I was (and am, probably) suffering from EVE overload, and I haven't been looking at the website for a bit, except to see whether it was running.
My ISP informed me that they were going to restart one of my servers to do some maintenance, but unfortunately informed me wrongly , and restarted the EMK server without me knowning, leading to the EMK processes stopping and not being started again for a few days. The maintenance is now complete, and the uploads are being processed again.
Should be steadily decreasing the backlog now. Last time I checked about 50000 / 2.5 hours.
UPDATE: Nothing has changed to the market scanner in the past weeks... I cannot imagine any reason for the errors except a change in the client. Which would be odd, since the other market sites are obviously working... Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
817
|
Posted - 2012.04.02 13:29:00 -
[524] - Quote
Thinking about how to calculate the top-10 market hubs again.
The naive method - just sum up all the buy orders value in that location, report on that. Then sum up all the sell order value, report on that. Maybe drop the outliers first.
The 5% method - Take the 5% sell price for each item, multiple by the total number of units for sale. This avoids the "1M ISK shuttle" from skewing the numbers. May as well do the same for the buy price.
Sum of the daily transactions - Use the history data, average the past 7 or 14 days, sum up the amount of ISK that changed hands. But this is only region-wide, not station/system.
Things I would like to see:
- Top market regions by ISK value, both top 10 buy-order regions, top 10 sell-order regions and top 10 combined value.
- Top systems by ISK value (so all of the stations in Jita, not just CNAP 4-4).
- Top stations by ISK value
You should track the top 20 locations in a table each day so you can present the top 5 or top 10 as a graph showing how they stack up over time.
I'm very curious to see how the various regions stack up against each other, and what the top 10 regions / systems are. |
Vaerah Vahrokha
Vahrokh Consulting
487
|
Posted - 2012.04.03 10:14:00 -
[525] - Quote
@Scrapyard
I noticed the backend used for this website has seems to have poor concurrency locking granularity. When you send some large queries (aggregates like the ones you wish above) it may take several seconds. Some database back ends lock the tables during those queries and therefore somebody trying to upload their data will be affected. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
116
|
Posted - 2012.04.03 16:16:00 -
[526] - Quote
The uploading process is pretty much unaffected by big queries (at least, as far a mysql is concerned), but the server itself might be a bit slow. Regardless, this is a query you will do at most once a day, and cache the results. That is acceptable in my opinion. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
819
|
Posted - 2012.04.04 05:37:00 -
[527] - Quote
Speaking of big, slow queries... the Consumer Price Index page takes forever.
http://www.evemarketeer.com/trade/consumerindex |
Scrapyard Bob
EVE University Ivy League
825
|
Posted - 2012.04.05 00:52:00 -
[528] - Quote
So, if I go to this link: http://www.evemarketeer.com/uploader
The "play" button only works about 1/2 the time at getting the market window to open up.
But if I go to: http://www.evemarketeer.com/uploader/display/all/orders
Then it mostly seems to work correctly.
When it breaks, it simply won't open the market window, but the timer sits and cycles through items and counts down. |
Scrapyard Bob
EVE University Ivy League
827
|
Posted - 2012.04.05 13:13:00 -
[529] - Quote
http://www.evemarketeer.com/statistics/uploads
Uploads are backed up again (129k and counting). |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
116
|
Posted - 2012.04.07 09:49:00 -
[530] - Quote
I finally found time to do something with EMK again, so I'll be looking into this today. And see if I cannot have those processes autostart. Developer/Creator of EVE Marketeer
|
|
Scrapyard Bob
EVE University Ivy League
835
|
Posted - 2012.04.07 11:44:00 -
[531] - Quote
Callean Drevus wrote:Update: Going to http://www.evemarketeer.com/uploader and clicking play, works for me each and every time. Is there anything specific you are doing before that? I just open the browser and go right to that page then click play. In addition on what machine are you running EVE, and what are it's settings?
Running in 1024x768 minimum setting windowed clients. Alt-B, EMK's market scanner is the home page, click play. It just seems like the countdown timer is a free running bicycle chain which never "catches" to open up the market window. Once it does latch on, it works fine.
EMD's page works flawlessly every time.
My guess, opening up the window causes a EVE API call, and clicking play button too soon might result in a second EVE API call before the 3-second timer resulting in a mute.
|
Vaerah Vahrokha
Vahrokh Consulting
514
|
Posted - 2012.04.08 21:23:00 -
[532] - Quote
Since the other day when the upload stopped working, I am unable to send prices any more. I have cache files delete enabled on startup. It deletes them but won't upload anything, I just see the windows open and close like in the past but nothing gets sent. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Scrapyard Bob
EVE University Ivy League
842
|
Posted - 2012.04.09 13:07:00 -
[533] - Quote
Vaerah Vahrokha wrote:Since the other day when the upload stopped working, I am unable to send prices any more. I have cache files delete enabled on startup. It deletes them but won't upload anything, I just see the windows open and close like in the past but nothing gets sent.
Check to see if it lost track of your upload keys. Try clearing cache for each junction point. If you have multiple junction points, make sure you fire up a EVE instance for each junction point (fixes the cache). Restart the uploader.
(The above works for me 99.5% of the time.) |
Vaerah Vahrokha
Vahrokh Consulting
516
|
Posted - 2012.04.09 21:48:00 -
[534] - Quote
Scrapyard Bob wrote:Vaerah Vahrokha wrote:Since the other day when the upload stopped working, I am unable to send prices any more. I have cache files delete enabled on startup. It deletes them but won't upload anything, I just see the windows open and close like in the past but nothing gets sent. Check to see if it lost track of your upload keys. Try clearing cache for each junction point. If you have multiple junction points, make sure you fire up a EVE instance for each junction point (fixes the cache). Restart the uploader. (The above works for me 99.5 of the time.)
I don't have junction points, I have a second computer I put only to do this sort of stuff (only 1 client ever runs in there).
I noticed on my main computer I can get never-upload-before items do get uploaded (the others won't, due to an issue Callean told earlier).
On the second computer it just won't upload anything ever. If I run my own made cache reader I can pull out everything easy, as this post can prove. The fun thing is, that second computer used to be the one that flawlessy sent everything up.
Is it possible that a recent patch created another "machonet" folder so the EvE Marketeer uploader does not delete the old folder files? If so, what should I manually delete to be sure I don't bork something like i.e. my settings? Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Scrapyard Bob
EVE University Ivy League
846
|
Posted - 2012.04.10 00:28:00 -
[535] - Quote
Vaerah Vahrokha wrote: If so, what should I manually delete to be sure I don't bork something like i.e. my settings?
Settings files on Win7 are under:
C:\Users\username\AppData\Local\CCP\EVE\c_program_files_(x86)_ccp_eve_tranquility\settings
You should always backup the settings folder regularly.
Cache folders are in the same directory tree and can be freely deleted (if the EVE clients are shutdown).
If you run junctions or multiple clients, then you'll have multiple folders under CCP\EVE, one for each path to the EVE executable. |
Gath Ra'nan
EVE University Ivy League
0
|
Posted - 2012.04.10 09:42:00 -
[536] - Quote
FYI,
Fatal error: Uncaught exception 'Predis_CommunicationException' with message 'Connection refused' in /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php:1520 Stack trace: #0 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(1470): Predis_Connection->onCommunicationException('Connection refu...', 111) #1 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(1446): Predis_Connection->redisStreamInitializer(Object(Predis_ConnectionParameters)) #2 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(1600): Predis_Connection->connect() #3 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(1551): Predis_Connection->getSocket() #4 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(1525): Predis_Connection->writeBytes('*2??$3??GET??$1...') #5 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(1534): Predis_Connection->writeCommand(Object(Predis_Commands_Get)) #6 /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php(203): Predis_Connection->executeCommand(Object(Predi in /var/www/www.evemarketeer.com/core/lib/redis/lib/Predis.php on line 1520
Great site, btw |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
117
|
Posted - 2012.04.10 14:39:00 -
[537] - Quote
Oh joy...
I'm starting to hate the server maintenance that seem to have to happen every week... It is fixed. Uploads will not be processed though, until I can restart those processes.
Regarding the uploader, it should always use the newest cache folder on startup, an completely ignore any others. Developer/Creator of EVE Marketeer
|
Styth spiting
Ion Corp. Citex Alliance
18
|
Posted - 2012.04.10 15:52:00 -
[538] - Quote
I wanted to find out if you are interested in another developer getting involved in this project. I've all ready spent roughly 40 - 60 hours working with the code and have a lot of ideas for new features (and performance ideas).
The problem I am running into though is that there is no data dump of default values (items for example) so even after getting everything setup on my dev server I cannot duplicate the existing site. Its basically a functioning site with no data (pretty important).
Some of the things I have been working on are:
- consolidating many of the configs into 1 primary config file.
- Installation instructions (mainly for myself, but available to everyone).
- Properly sized icons (image icons are way to big for display withing the sites layout).
- Item database. Adding aucutal item data, not just item market data.
- Items reprocessed amounts.
- css / html optimization.
- various calculators and tools for players to use (ore to mineral converter using market data for example.
So yeah, if you want a kick ass developer helping out send me a message and we can get some discussion going. I love the project (and the existing code is pretty sexy) and would love to get involved with it. |
Snarf Aldes
University of Caille Gallente Federation
1
|
Posted - 2012.04.10 18:46:00 -
[539] - Quote
I think i found an issue with the unified uploader.
It seems that if one of the endpoints has issues, and the queue fills to 50, uploads to the properly working endpoints also stalls. It looks like uploads only happen as long as none of the queues are filled to 50.
Creator of Eve Addicts |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
2
|
Posted - 2012.04.13 16:53:00 -
[540] - Quote
This may be related to the uploads not getting processed, but this appears to be returning nothing:
http://www.evemarketeer.com/api/recent_updates/5/xml
If uploads still aren't processing, that would make sense. But it is a bummer for me, trying to steal your data :) EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
|
Vaerah Vahrokha
Vahrokh Consulting
567
|
Posted - 2012.04.14 00:37:00 -
[541] - Quote
Website completely dies in US prime time. Some times it gives some message like: "too many connections" other times like today it just goes with a long pause then "connection refused", "connection timeout" and similar. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Dalden V
Blue Lounge Industries
1
|
Posted - 2012.04.14 12:54:00 -
[542] - Quote
Vaerah Vahrokha wrote:Website completely dies in US prime time. Some times it gives some message like: "too many connections" other times like today it just goes with a long pause then "connection refused", "connection timeout" and similar.
Thanks Vaerah. I was wondering if it was just me that cannot access the site :). Today I'm constantly getting connection timeouts. |
plan q
Interstellar eXodus BricK sQuAD.
0
|
Posted - 2012.04.14 14:22:00 -
[543] - Quote
ah pretty sure it has been down since 2012/04/13 20:35:43 no api calls since then :D |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
117
|
Posted - 2012.04.14 15:09:00 -
[544] - Quote
Balls, it seems the site either crashed, or someone at my datacenter did something incredibly stupid, since my server died (presumably) without reason some 14 hours ago. I'm currently investigating this issue. Developer/Creator of EVE Marketeer
|
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 15:49:00 -
[545] - Quote
Interesting.. ~ breaks the forum.. big time.. :/ |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 15:49:00 -
[546] - Quote
-snip- accidental post |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 15:49:00 -
[547] - Quote
Callean Drevus wrote:How about this? http://chubaka.dyndns.biz/~ci/evemarketeer_stats.png (I just grabbed the code from the IPB Forums site and stuck it in there using your theme.) http://chubaka.dyndns.biz/~ci/stats_code.txt (if you like it, there is the code/links to images. I think they are the famfamfam silk icon ones though)
Your links are broken;
* dig chubaka.dyndns.biz +short 35.32.132.87 * ping -c 1 35.32.132.87 PING 35.32.132.87 (35.32.132.87) 56(84) bytes of data. --- 35.32.132.87 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 10000ms
Sounds like you need to fixy your dynamic dns service? :) |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
117
|
Posted - 2012.04.14 16:45:00 -
[548] - Quote
I didn't really understand what you meant to say...
The links you provided link only to a 404 page not found.
In regards to the server being down. The sudden move of my server was cancelled, but it doesn't seem like that will bring it up again quickly, so I can only ask you to bear with it for the moment. Developer/Creator of EVE Marketeer
|
Zifrian
Licentia Ex Vereor Intrepid Crossing
246
|
Posted - 2012.04.14 19:10:00 -
[549] - Quote
I'm sure it's all related to your current posts, but the following address: http://www.evemarketeer.com/api/averages/10000002/30/620/xml
Gave the following error: "Unable to connect to the remote server
Not sure if that helps
Thanks for looking into it. Maximze your Industry Potential! - Get EVE Isk per Hour! |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 19:13:00 -
[550] - Quote
Callean Drevus wrote:I didn't really understand what you meant to say...
The links you provided link only to a 404 page not found.
In regards to the server being down. The sudden move of my server was cancelled, but it doesn't seem like that will bring it up again quickly, so I can only ask you to bear with it for the moment. Something is really wrong with your routing mister. It looks like some sort of networking issue :(
Edit: Are you hosting your website on the same server that's doing DNS services? |
|
Desmont McCallock
163
|
Posted - 2012.04.14 20:34:00 -
[551] - Quote
Do you all fail on reading that his server is down? |
Aineko Macx
Royal Amarr Institute Amarr Empire
172
|
Posted - 2012.04.15 15:57:00 -
[552] - Quote
- When doing a history query of multiple items, the API is only returning data for the first typeID specified, ignoring the others - For some reason, the history API is throwing an error on typeID 4368. |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.15 19:15:00 -
[553] - Quote
Desmont McCallock wrote:Do you all fail on reading that his server is down? I can read.
dig www.evemarketeer.com ns ;; Got bad packet: extra input data 101 bytes 2d 0f 81 80 00 01 00 00 00 01 00 00 03 77 77 77 -............www 0c 65 76 65 6d 61 72 6b 65 74 65 65 72 03 63 6f .evemarketeer.co 6d 00 00 02 00 01 c0 10 00 06 00 01 00 00 38 40 m.............8@ 00 33 00 00 0a 68 6f 73 74 6d 61 73 74 65 72 0c .3...hostmaster. 65 76 65 6d 61 72 6b 65 74 65 65 72 03 63 6f 6d evemarketeer.com 00 77 df 4d 38 00 00 38 40 00 00 0e 10 00 12 75 [email protected] 00 00 00 38 40 ...8@
^ Get a better DNS resolver?
dig www.evemarketeer.com ns @ns1.coronum.nl +short
; <<>> DiG 9.7.3 <<>> www.evemarketeer.com ns @ns1.coronum.nl +short ;; global options: +cmd ;; connection timed out; no servers could be reached
dig evemarketeer.com a +short 89.31.102.172 (redirects) dig www.evemarketeer.com a +short 50.28.89.81 (site loads)
Why not change the @ DNS to point to the new ip address? I think you should get a better name server resolution, there are several out there. I'd suggest; http://zoneedit.com
Or failing that.. http://freedns.afraid.org/ :)
Edit: Looks like your on liquid web; http://www.liquidweb.com/dedicated/single-processor.html
Why not switch to a more reliable VPS provider like linode? Just curious. |
Vaerah Vahrokha
Vahrokh Consulting
584
|
Posted - 2012.04.17 00:35:00 -
[554] - Quote
While accessing the website I am getting blank pages that say:
502 Bad Gateway
nginx/0.7.65
When it does not do that, it takes 30 seconds to show a page (hilariously, it says the query took 0.8 secs to run). Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
LifeHatesMe
SKULLDOGS RED.OverLord
5
|
Posted - 2012.04.17 01:04:00 -
[555] - Quote
Vaerah Vahrokha wrote:While accessing the website I am getting blank pages that say:
502 Bad Gateway
nginx/0.7.65
When it does not do that, it takes 30 seconds to show a page (hilariously, it says the query took 0.8 secs to run). NGINX is a lightweight http daemon service.
Bad Gateway seems to mean that how it's connected with the PHP parser is not syncing properly, there are many reasons this happens. I don't know your config, are you using nginx for everything? A reverse proxy with apache? or something else?
I still think you could get a better host. The whole reason people use nginx in the first place is because they lack enough ram to use Apache reliably. |
Philderbeast
M.I.M.M.S Apocalypse Now.
12
|
Posted - 2012.04.17 05:14:00 -
[556] - Quote
LifeHatesMe wrote:Vaerah Vahrokha wrote:While accessing the website I am getting blank pages that say:
502 Bad Gateway
nginx/0.7.65
When it does not do that, it takes 30 seconds to show a page (hilariously, it says the query took 0.8 secs to run). NGINX is a lightweight http daemon service. Bad Gateway seems to mean that how it's connected with the PHP parser is not syncing properly, there are many reasons this happens. I don't know your config, are you using nginx for everything? A reverse proxy with apache? or something else? I still think you could get a better host. The whole reason people use nginx in the first place is because they lack enough ram to use Apache reliably.
not so much they lack the ram, but nginx is faster at serving static content then apache and scales up much better then apache. infact many large sites use it like face book as it scales much better. |
Scrapyard Bob
EVE University Ivy League
882
|
Posted - 2012.04.19 17:32:00 -
[557] - Quote
Prices are now about 7 days old according to the little green/yellow/orange dots.
http://www.evemarketeer.com/item/group_summary/18
There's also close to 100k in queued uploads. |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
3
|
Posted - 2012.04.20 01:45:00 -
[558] - Quote
LifeHatesMe wrote:Callean Drevus wrote:How about this? http://chubaka.dyndns.biz/~ci/evemarketeer_stats.png (I just grabbed the code from the IPB Forums site and stuck it in there using your theme.) http://chubaka.dyndns.biz/~ci/stats_code.txt (if you like it, there is the code/links to images. I think they are the famfamfam silk icon ones though)
Your links are broken; * dig chubaka.dyndns.biz +short 35.32.132.87 * ping -c 1 35.32.132.87 PING 35.32.132.87 (35.32.132.87) 56(84) bytes of data. --- 35.32.132.87 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 10000ms Sounds like you need to fixy your dynamic dns service? :) That is from a long time ago. Well, that used to exist. It was on a computer sitting in my dorm room. As for the packet loss, my router is set not to respond to pings. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
118
|
Posted - 2012.04.21 20:21:00 -
[559] - Quote
LifeHatesMe wrote:Why not switch to a more reliable VPS provider like linode? Just curious.
Because Liquidweb is better, incredibly less expensive and faster. Those are just matters of opinion, but they count for a lot with me.
The whole reason people use Nginx, is that there is little reason to load PHP into memory just to serve an image. When using apache, there is little reason not to use mod_php, so...
In general, I just feel nginx is the better of the two for my purposes.
As you might have noted, the server is now up again, though it still seems to not be processing any uploads (or not saving or showing it anyway). I'm going to investigate that now.
I'm sorry for my inability to really take care of the website, but there are just more important things in my life right now. Developer/Creator of EVE Marketeer
|
Vaerah Vahrokha
Vahrokh Consulting
631
|
Posted - 2012.04.22 19:17:00 -
[560] - Quote
Today for some reason the upload started working again even on the computer where it would not work. But I still have one odd thing.
I upload very fresh data, I get your website to take it (I see it in my upload stats and on the client side uploader) but I always get about 2 days older history on your website than the history I sent you. I.e. I sent history for Nocxium up to Apr 20 yet on the website OHLC data dump it only covered up to Apr 18. 4 days old data is quite useless. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
|
Vaerah Vahrokha
Vahrokh Consulting
631
|
Posted - 2012.04.22 22:29:00 -
[561] - Quote
Another noobie question. For the first ime I have looked at the market orders. They website says they were updated 1 minute before I read them.
Yet in game I see Nocx @Jita CNAP at 985 or so on sell orders, but I see a number of website sell orders @Jita CNAP on the website for about 531 ISK pu before the 985 ones. What gives? Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Andy DelGardo
Hedion University Amarr Empire
3
|
Posted - 2012.04.23 12:25:00 -
[562] - Quote
Hi,
small bug report: I managed to break the group_summary lists for my account like "http://www.evemarketeer.com/item/group_summary/1119" by adding a 6th kinda unusual region, now my summary page is complete empty and i also cant remove/change the regions anymore since the button is missing. I tried deleting my cookies, but seems this is stored on the account? Is there a way to fix this view without recreating a new account?
thx Andy |
Controller Vrelk
New Eden Regimental Navy Rebel Alliance of New Eden
3
|
Posted - 2012.04.25 15:56:00 -
[563] - Quote
I've been getting a 502 error for the past day. |
Zifrian
Licentia Ex Vereor Intrepid Crossing
258
|
Posted - 2012.04.28 15:48:00 -
[564] - Quote
My users are reporting a locking error. I'll put in a work around for timing out long requests but this was the response I got to the following query:
http://www.evemarketeer.com/api/averages/10000043/90/34_35_36/xml
Warning: mysql_select_db() expects parameter 2 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 38 Warning: mysql_ping() expects parameter 1 to be resource, boolean given in /var/www/www.evemarketeer.com/core/lib/cordb/cordb.php on line 29 Database connection #0 failed
Hope that helps. Thanks Maximze your Industry Potential! - Get EVE Isk per Hour! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
119
|
Posted - 2012.04.28 15:52:00 -
[565] - Quote
Yes, the server was getting timeouts, and now mysql is angry I think. Developer/Creator of EVE Marketeer
|
Scrapyard Bob
EVE University Ivy League
899
|
Posted - 2012.04.29 21:09:00 -
[566] - Quote
On the unified uploader - If a particular end-point is causing problems (the queue hits 50), then the algorithm should simply skip adding any more items to that end-point's queue until it drops below 50 again. That would probably take care of the issue that if a particular end point is having trouble, the rest of the end points suffer because the upload tool slows down.
Just drop any items which would overflow the queue for that endpoint. If the endpoint is having trouble, to the point that the queue is full, it's probably not going to catch up for a while so no point queuing up additional items. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
120
|
Posted - 2012.05.06 08:48:00 -
[567] - Quote
@Scrapyard: That is what should be happening right now. It drops any items above 50, which is why the queue remains at 50. The problem is more that if it is slow enough to grow to 50, it will probably always remain there, as items are added faster than they are uploaded.
As uploads are handled in different threads, the only thing I can see having any effect on upload speed for other endpoints is the outbound bandwidth of your connection. Have you actually seen the uploader get a lot slower if one of the endpoints (AKA EVE Marketeer for the past week) is in trouble?
@Everyone: I'm sorry I am unable to fix the problems. It just grew a bit above my head, and I have little ideas on how to fix it except to do a total reset of the data involved, that might be painful, but at least the website would be working again. Developer/Creator of EVE Marketeer
|
Andy DelGardo
Hedion University Amarr Empire
4
|
Posted - 2012.05.06 11:11:00 -
[568] - Quote
Callean Drevus wrote: @Everyone: I'm sorry I am unable to fix the problems. It just grew a bit above my head, and I have little ideas on how to fix it except to do a total reset of the data involved, that might be painful, but at least the website would be working again.
I have no problems with a reset, since i personally use the site for short-term price observations, so i'm only interested in data that is at least 7 days old. If mr. Bob continues his upload's the site will be back at normal in no time :p
Only the SVR data will be problematic for the first month.
bye Andy |
Vaerah Vahrokha
Vahrokh Consulting
720
|
Posted - 2012.05.08 09:05:00 -
[569] - Quote
Callean Drevus wrote: @Everyone: I'm sorry I am unable to fix the problems. It just grew a bit above my head, and I have little ideas on how to fix it except to do a total reset of the data involved, that might be painful, but at least the website would be working again.
Does it mean that you are going to lose all the history from 2009? It's one of the highest value your website provides.
If you can't save it, could you backup prices history in some text format and make it available to download? I'll gladly go dig it by hand if the alternative is to lose all. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
120
|
Posted - 2012.05.13 22:09:00 -
[570] - Quote
Vaerah Vahrokha wrote:Callean Drevus wrote: @Everyone: I'm sorry I am unable to fix the problems. It just grew a bit above my head, and I have little ideas on how to fix it except to do a total reset of the data involved, that might be painful, but at least the website would be working again.
Does it mean that you are going to lose all the history from 2009? It's one of the highest value your website provides. If you can't save it, could you backup prices history in some text format and make it available to download? I'll gladly go dig it by hand if the alternative is to lose all.
Price history was happily not lost, as I had just a week ago made a backup dump of all the statistical information. That was pretty good timing in hindsight.
You can reach this dump too, by going to http://www.evemarketeer.com/var/dump
I hope ot periodically update that with the new data, so it will never be suddenly lost.
On another note, the website is FUNCTIONAL again, accepting uploads, and updating. God (and you all) knows it took me long enough to fix it.
For now, I'll try to keep it operational, and free of stupid bugs, but adding new features will be on the backburner for a bit, until I get more free time again at the beginning of September. Sorry.
PS: Though I hopefully will get around to implementing UUF messages (for EMUU) and retrieving messages from EMDR. After all, EVEMon will be the most terrific datasource we ever had. Developer/Creator of EVE Marketeer
|
|
Andy DelGardo
Hedion University Amarr Empire
27
|
Posted - 2012.05.25 09:32:00 -
[571] - Quote
Getting this error if i try to use the api: "http://www.evemarketeer.com/api/region/xml/10000043"
"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 81 bytes) in /var/www/www.evemarketeer.com/core/mod/content/yapeal.php on line 115 " |
Devai Starchild
Shattered Planet Initiative
4
|
Posted - 2012.05.25 23:57:00 -
[572] - Quote
Currently will not register any of my character data,
When trying to view anything under the character tab, it shows this error:
MYSQL ERROR OCCURRED: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY lft ASC' at line 1
You have the best marketing site out there. Please don't let it die. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
124
|
Posted - 2012.06.03 14:26:00 -
[573] - Quote
Holy ******* hell. This was yapeal... I should've checked when I fixed the database, but the gathering of character, corporation and eve data is turned off by default when you generate the yapeal database (undoubtedly to protect us from too much data). In this case however, the only thing it was doing was preventing any useful data from being gathered at all. Developer/Creator of EVE Marketeer
|
Dragonaire
Corax. The Big Dirty
44
|
Posted - 2012.06.03 19:44:00 -
[574] - Quote
Callean Drevus wrote:Holy ******* hell. This was yapeal... I should've checked when I fixed the database, but the gathering of character, corporation and eve data is turned off by default when you generate the yapeal database (undoubtedly to protect us from too much data). In this case however, the only thing it was doing was preventing any useful data from being gathered at all. Just for future reference I thought I'd let everyone know I changed the default so all the sections are on now. One of those legacy things that I'd never really thought about so never looked if it was still really needed Got a new post up about it on the Yapeal thread. Sorry for the trouble it caused
Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Andy DelGardo
Hedion University Amarr Empire
65
|
Posted - 2012.06.14 15:36:00 -
[575] - Quote
The uploads seem not being processed anymore? Latest date i get is "2012-06-09" even if i upload data and wait couple of hours. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
125
|
Posted - 2012.06.15 06:56:00 -
[576] - Quote
Yeah, the queue was filling up. It seems the processing threads had stopped, which I only noticed today because of another error. It seems there is a somewhat big backlog, but it will eventually all be processed now. Developer/Creator of EVE Marketeer
|
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
6
|
Posted - 2012.06.15 14:09:00 -
[577] - Quote
I may be able to point to EVE Addicts as somewhat of a recent "case study". They were running into the socket limit, seeing higher resource util on uploads, and generally struggling to keep up with the new EVEMon uploads. After disabling regular uploads and switching to EMDR, they found themselves dealing with a tiny fraction of the load they were before.
The reasons are numerous:
- Accepting many HTTP uploads in a short period of time involves dealing with that many HTTP connections.
- Rapidly opening and closing TCP sockets in those proportions can really kill a machine that is doing too many other things, or isn't configured just right (sysctl's ipv4 and tcp directives are huge here).
- EMDR's ZeroMQ underpinnings use one or two TCP connections at any time, max. The socket is left open, there is no constant opening/closing, and everything is compressed.
- 100% of EMDR's messages are compressed, unlike what you're dealing with in EMK.
I realize that you've got your own uploader and perhaps a slightly different set of people uploading, but the EVEMon flood sends us anywhere from 1-3 million uploads a day. A potential compromise may be to work with me to point your EMK format uploads at EMDR, then consume from EMDR. This would shift the burden, and potentially free your servers up somewhat. It would also be 100% revertible. You would then only need to take our sample consumer, add some concurrency, and parse from there.
Get in touch with me (GTalk, email, #emdr on irc.coldfront.net, whatever) if any of this appeals to you. If not, completely understand, let me know if we can help in any other ways.
EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Shinya Shazih
Bene Gesserit ChapterHouse Sanctuary Pact
0
|
Posted - 2012.06.15 22:28:00 -
[578] - Quote
Site seems broken? I get the same price for three regions
=ImportXML("http://api.eve-marketdata.com/api/importxml_prices2.xml?char_name=shinya shazih&buysell=b®ion_id=10000002&type_ids="&JOIN(",",$D4:$D30); "/emd/price")
It worked before, anything changed? |
Golden Gnu
EVE University Ivy League
61
|
Posted - 2012.06.16 11:57:00 -
[579] - Quote
First: THANK YOU! EveMarketeer is pure awesomeness!
Second: It looks like your site is being very slow. Download is the meaning of life, upload is the meaning of intelligent life http://eve.nikr.net - home of jEveAssets |
Andy DelGardo
Hedion University Amarr Empire
65
|
Posted - 2012.06.17 12:02:00 -
[580] - Quote
"special: separate with _ to query multiple id's at once, max 25"
This seems not to work for the "History" api call? Is this a api or documentation error?
Thx Andy |
|
Vaerah Vahrokha
Vahrokh Consulting
1345
|
Posted - 2012.06.19 16:21:00 -
[581] - Quote
Even uploading history I can't get prices newer than June 15. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Drachiel
Mercury LLC
11
|
Posted - 2012.06.21 14:31:00 -
[582] - Quote
A service. Formerly a B, since my .bat that pulls market data from my favorite items into Excel every 12 hours sometimes pull corrupt / bad data from the site. Hasn't happened in a few months, so good dev progress. |
Drachiel
Mercury LLC
11
|
Posted - 2012.06.24 23:17:00 -
[583] - Quote
Back to a B again, 502 errors for several days in a row. |
anishamora
Garoun Investment Bank Gallente Federation
3
|
Posted - 2012.06.26 10:21:00 -
[584] - Quote
Drachiel wrote:Back to a B again, 502 errors for several days in a row.
Same here :( |
Alaya Carrier
Republic Military School Minmatar Republic
41
|
Posted - 2012.06.27 20:16:00 -
[585] - Quote
Website totally died for me, since days.
Wish the Developer made the sources available for someone else to keep running this fantastic, super website we love. |
Jack Arc
Deep Core Mining Inc. Caldari State
0
|
Posted - 2012.06.27 21:29:00 -
[586] - Quote
https://bitbucket.org/aeolun/eve-marketeer |
Styth spiting
Ion Corp. NightSong Directorate
32
|
Posted - 2012.06.30 05:13:00 -
[587] - Quote
Alaya Carrier wrote:Website totally died for me, since days.
Wish the Developer made the sources available for someone else to keep running this fantastic, super website we love. The source code has been made available for quite a while now (just not publicly advertised).
As Jack posted it is located https://bitbucket.org/aeolun/eve-marketeer.
The repository includes all the php files and some of the database structure, but none of the data. It does use the default crucible database tables for most of the items, ships, maps etc though so you can get 99% of it functioning. Having worked with it for several months now the biggest obstacle was re-creating the data that was needed for the app its self (default values, login data, cron jobs, etc).
Callean was awesome enough to give me access to the missing data and have been able to successfully re-create a development version for myself which will be showing up as a forked project of eve-marketeer. The hope is the fork will make available more features that he will be able to fork into eve-marketeer.com as he sees fit. |
Alaya Carrier
Republic Military School Minmatar Republic
46
|
Posted - 2012.07.01 19:17:00 -
[588] - Quote
Yes please. That website was a gem. It'd be really sad to lose it. |
Drachiel
Mercury LLC
11
|
Posted - 2012.07.02 13:53:00 -
[589] - Quote
It's running again, for what it's worth. |
Alaya Carrier
Republic Military School Minmatar Republic
49
|
Posted - 2012.07.03 12:54:00 -
[590] - Quote
Drachiel wrote:It's running again, for what it's worth.
To me it's worth a lot. Thanks! |
|
Dunan Sanis
State Protectorate Caldari State
0
|
Posted - 2012.07.06 14:28:00 -
[591] - Quote
WoW the site is nearly dead, what a shame...
Would it be hard to switch to EMDR for data and ease this poor server pain?
Also where can we donate for a better server?
Thank you for your extraordinary tool ! |
Lyron-Baktos
Selective Pressure Rote Kapelle
312
|
Posted - 2012.08.22 15:39:00 -
[592] - Quote
so, no more support for this site? Selective Pressure [FOVRA] is now recruiting! https://forums.eveonline.com/default.aspx?g=posts&m=1797934#post1797934 |
Desmont McCallock
213
|
Posted - 2012.08.22 17:07:00 -
[593] - Quote
I tried (about a month ago) to poke Callean Drevus to support the format, that EMDR/EVEMon is using, in order for his site to benefit from the excess data flow. The response I got was that we had lost interest in EVE (at least for the moment) as RL comes first (totally acceptable). Until things change in his RL don't expect anything. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
127
|
Posted - 2012.09.01 14:51:00 -
[594] - Quote
Well, change has come, and with the change, a surprising increase in motivation.
Let me explain in some more detail what happened (TEXT WALL ALERT):
About 4 months or so ago, I was happily working on my Job, and on EVE Marketeer, when I got an offer for another job, which was awesome. The catch was, that to get it, I had to work part-time for my would be new employer while I waited for my current contract to end. Needless to say, this imposed quite a lot on my spare time, as I was essentially working about 70 hours a week in total.
70 hours of work would've left some time for EVE Marketeer, but I found that after doing the programming work of two jobs, I just couldn't do anything more than just stare at a TV screen, watch a movie/series, or play a simple but addictive game like League of Legends, anything that didn't actually require me to think too much was fine.
At that point, the decision to drop EMK for the moment was made, as I had run into a few walls with the website anyway, and I couldn't really continue with expanding it without a significant rewrite. This was not the EMDR part, which was easy to add, but I just couldn't motivate myself to add it to the website when I knew I was going to totally rewrite it anyway once I returned to business. Besides, with the current processing mechanism, EMK would positively DIE the moment I turned open that valve :P
END OF TEXT WALL
Anyway, that is the long story. I'm now back to one job, which allows me to work from home even, so I should have much more opportunities to do some useful EMK work :) and hopefully supply you with (more) awesome stuff in the future again.
I'll be modifying the front post to let everyone seeing the topic know I'm back, and I'm very anxious to hear from you what exactly you used most on the current website, and what you would like to see after a rewrite. Anything goes, even things you think are impossible! Developer/Creator of EVE Marketeer
|
Benign Sage
Rock Ballz INC.
13
|
Posted - 2012.09.01 16:02:00 -
[595] - Quote
Glad to see you are back! Soon as you get things tuned up I will be wanting to integrate my website into your API! Very excited! My EVE API-á https://www.myeveapi.com |
Domi Etymology
C0NVICTED HOLDING C0NVICTED
0
|
Posted - 2012.09.09 09:09:00 -
[596] - Quote
Great news to hear you're back, Callean. I'm eagerly looking forward to seeing Eve Marketeer retake it's place as the best eve market site out there :)
Also, especially hoping to see implemented that trade route finder that was initially looked at so long ago :) |
TheSmokingHertog
TALIBAN EXPRESS
63
|
Posted - 2012.09.11 18:14:00 -
[597] - Quote
Have fun in your new job! :) |
Makro SSRI
dat corp
3
|
Posted - 2012.09.11 20:10:00 -
[598] - Quote
Looking forward to new awesome stuff! |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
128
|
Posted - 2012.09.12 10:39:00 -
[599] - Quote
Happy to see you all still following :) I hope to be able to show something soon. Developer/Creator of EVE Marketeer
|
Leolie Aylet
Center for Advanced Studies Gallente Federation
1
|
Posted - 2012.09.15 08:51:00 -
[600] - Quote
Any news ? |
|
Domi Etymology
C0NVICTED HOLDING C0NVICTED
0
|
Posted - 2012.10.07 01:43:00 -
[601] - Quote
How's things goin here Callean? I'm sure you've got a bunch of us eagerly awaiting the return of marketeer :) |
TheSmokingHertog
TALIBAN EXPRESS
64
|
Posted - 2012.10.12 00:35:00 -
[602] - Quote
Would be great to get it back in business indeed. |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
130
|
Posted - 2012.10.21 12:54:00 -
[603] - Quote
Excuse me, I had a little less time than I thought I'd have, so it was taking a bit longer than I anticipated. This made me feel ashamed, since I said I'd have something by the end of September, so I felt hesitant about showing my face here until I actually had something :/
When I decided to built EMK again I wanted to do that with different technology, so that I might actually learn from it again, while preventing some of the mistakes I made previously. I originally wanted to build it in Node.JS, which was awesome, but which was so alien to me that I felt I wouldn't be able to deliver anything within a decent timespan, nor anything of the quality I really wanted. So by the end of september, I had something, but I had to scrap it because it wasn't decent enough to show anyone.
I'm now building in Python, and I have a much better feeling about it. I'm currently focusing on the backend, which will hopefully allow API calls first, then allow me to build onto that to give the website a front-end. I'm planning to use websockets for the frontend, which will hopefully make it insanely responsive compared to a traditional webapp (only it won't work in the IGB, so it' ll have to fallback to simple AJAX there). Developer/Creator of EVE Marketeer
|
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.10.21 20:18:00 -
[604] - Quote
I'll just point this out, but if you're working in Python, I'd encourage you to combine forces with these guys: http://www.element-43.com/
They're making great progress, and have a good number of people involved.
Source: https://github.com/EVE-Tools/element43
IRC Channel: #element43 on ColdFront EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
130
|
Posted - 2012.10.21 20:55:00 -
[605] - Quote
Hey, thanks for that. It looks pretty good. I'll see whether anything comes of it.
It looks good, though not as good as I would've liked.
Problem with joining an already existing website is that I'll have to work together with people whom might not share my vision of what it could/should be, but it would be a great help to not be the single person responsible for keeping the website up. In addition, competition is good, but too much competition makes it very difficult to choose. Adding EMK to the mix again brings the count of market websites to a number above 5 :P Developer/Creator of EVE Marketeer
|
Cholly Chi
Pator Tech School Minmatar Republic
0
|
Posted - 2012.10.28 15:49:00 -
[606] - Quote
The job well done is worth waiting for - take your time, and best of luck:) |
|
|
|
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 .. 21 :: [one page] |