BOINC@AUSTRALIA FORUM

Retired BOINC Projects => THESKYNET POGS => Topic started by: Dingo on August 04, 2012, 12:27:05 AM

Title: Testing of BOINC version of theSkyNet
Post by: Dingo on August 04, 2012, 12:27:05 AM
I am one of the testers for theSkyNet in BOINC which is being conducted now.  There is no credit for the testing but I like being a Alpha Tester especially for this first Australian BOINC project. 

They do not have any vacancies for more testers during this Alpha stage but I will be sending out an email to the Team when it comes to "Volume Testing" of the project sometime in the future.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Mike Mitchell on August 04, 2012, 01:10:14 AM
Thanks for that Dingo.  :thanks1:
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 04, 2012, 01:44:11 AM
Cool can't wait.   :thumbsup:

Mmm, I may be getting ahead of myself a little (or maybe alot) "but" any chance of SkyNet GPU maybe? err one day?

Now that would be something I could really get my teeth into (what's left of em' ) dang sweet tooth.. ;D
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 04, 2012, 04:41:09 AM
 :thumbsup:
:thanks1:
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 04, 2012, 12:19:04 PM
Quote from: JugNut on August 04, 2012, 01:44:11 AM
Cool can't wait.   :thumbsup:

Mmm, I may be getting ahead of myself a little (or maybe alot) "but" any chance of SkyNet GPU maybe? err one day?

Now that would be something I could really get my teeth into (what's left of em' ) dang sweet tooth.. ;D

It is very early days and they will just be trying to get the system to work first with CPU.  At the moment I have not had any validated, I have had a couple "Deleted by the server" It is minimum quorum   2
initial replication   3 for work units and only about 40 users so it is taking a while.  The work units currently run between 2 and 15 hours. 

EDIT While I was typing this I got three validated.  They have an average of 3.23 credits per hour, but the credits will not go anywhere.
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 04, 2012, 12:42:16 PM
Thanks Dingo, I thought that might be the case. (but It was nice to dream)

Thanks for keeping us informed..
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 10, 2012, 12:22:48 PM
The project is now open for more users now in Beta stage.  If you would like to test this first Australian BOINC project please use the url below in BOINC and join the Team.  Task run about 10- 17 hours and credits are changed to about 26 credits per task per hour.

 http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/ (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/)
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 10, 2012, 01:59:52 PM
On it, thanks for the email.  :thumbsup:
Great to finally have an Aussie BOINC project. :dance:
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 10, 2012, 03:03:17 PM
Like wise..  :thumbsup:
Title: Re: Testing of BOINC version of theSkyNet
Post by: tazzduke on August 10, 2012, 10:27:20 PM
Okay jumped onboard with this project, but wouldnt you know it, NRG decide to release a reasonable amount of Workunits, also having a bit of fun with the NFS challenge as well, so I got one workunit crunching at the moment.

AusTerror has jumped onboard with the NFS challenge as well as kingdad66, which is good to see.

Seti challenge is going well, albeit my pendings are increasing by the minute

After cleared out the workcache and the NFS challenge has finished will get a bit more crunching happening, till then

Happy Crunching
Tazzduke
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 11, 2012, 02:28:43 AM
I joined but can't get work yet.
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 11, 2012, 03:28:44 AM
That's because my computer has hoovered up 20,000 galaxies like a gigantic black hole. biggrin

I guess an influx of new contributors has exhausted the work that was provided. It's a very new project so the admin possibly has to get used to not letting the hopper go dry with more contributors to feed. Or perhaps he wants to check the batch to see if everything is running correctly before releasing more. It's 1.25AM now in Perth, I noticed he posted on the forum last weekend in the afternoon, so hopefully there will be some work within the next day.

I was a little miffed myself after I cleared the decks a bit to go to town and then found there were no tasks left. Although I don't need many as they take 10-12 hours each on my computer. I've still got 11 tasks in my cache so am OK for a while. I completed 7 with no troubles, real smooth crunching and low memory usage. :thumbsup:

Even had one validated so I'm on the board. v:

Go, go theSkyNet POGS.:yahoo:
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 11, 2012, 07:41:19 AM


This is good, everything works.  Picked up 20 WUs just after midnight.   :aus1: at LAST!!!!!!
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 11, 2012, 12:14:44 PM
 :cheer1: "Yea"  1st points.   409 of the little  beauties for one validated WU .  :cheer1:

v:

Makes me 3rd in team by the chart at Skynet. After Dingo & Kashi..

http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/team_members.php?teamid=6&offset=0&sort_by=total_credit (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/team_members.php?teamid=6&offset=0&sort_by=total_credit)

err I hope that link works without you having to log-in first? Anyone?
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 11, 2012, 12:44:00 PM
Most of the tasks I did in the beginning only granted 3 credits per hour so won't take long to overtake me.  Not much work around on the project, had a lot of new users and they must have gotten a lot of tasks.   Kevin, the Project Developer has been very good at answering messages on the Message board and should let us know when the next galaxies will be ready.
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 11, 2012, 01:10:56 PM
Yeah you would have had about 15-20K for valid tasks by now if the current rate had applied earlier. Probably had invalids too which isn't happening now thanks to the all the testing and refinement.
So great work on the testing Dingo, thank you.  :congrats    :congrats

Yes I can see the team without logging in JugNut. You should sprint past me soon, but not quite yet. ;D
Title: Re: Testing of BOINC version of theSkyNet
Post by: Mike Mitchell on August 12, 2012, 12:55:57 AM
I just joined the project, and added the team, got one work unit. I'll run that and see what it's like before making plans. I'm only the 300th project member.

Hmm, I have an idea for the next AA.  :wink
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 12, 2012, 07:30:32 AM
I got 4 but big problems ... NO CHECKPOINTS????? I did my normal weekly PM on that machine and restarted at 0 time. No can do on 14-15 hour jobs. Any one getting checkpoints?  :hbang:
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 12, 2012, 08:51:17 AM
Checkpoints?  Was crunching 6 WUs that were between 4 and 75 percent complete.  
Had to power down for a couple of hours and when I restarted BOINC all the work units showed zero percent complete.
Now the weird bit -  After a couple of minutes four of the WUs shot back up to the percent complete stage they were at when I turned the machine off.  The other two did not.

I have to do the same thing later today.  I will take much more notice of the result so I can get onto the forum for answers if needs be.

EDIT:  Visit to the Forum shows problems with Windows while Linux and MACs work OK.   It is under investigation at this time.
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 12, 2012, 12:25:29 PM
Thanks for the update lawry.  I've had similar problems as well.  As I may have mentioned before I have two i7's one is 100% stable & one is at best 85% stable & much to much a risk on 12hr+ non check-pointing WU's.     I've stopped crunching on it & have other work that needs crunching also.  So while i'll keep a few core's with the light on, most of my effort will have to resume in a few days time. (hopefully when the check-pointing issue is fixed)

Cheers ..JN..
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 12, 2012, 12:29:30 PM
Yep, definately not checkpointing Windows. Have set it to no new tasks until I see something in the form.

HOT here and not much crunching going on. 106F

:aus1: :cheers: :US
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 12, 2012, 12:35:29 PM
Sorry guys I thought it was fixed for all Operating Systems.   :boom:
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 12, 2012, 01:53:37 PM
No harm no foul, Dingo. I'll keep watching.  :bloodshot
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 12, 2012, 02:10:13 PM
I had one task restart from a checkpoint successfully. "Leave applications in memory while suspended" was turned off when my preferences accidentally got reset. Have not tried exiting BOINC or rebooting though. Restart at fit 24 is shown in Stderr output:
"04:33:47 (4296): wrapper: running fit_sed (24 filters.dat observations.dat)
wrapper: starting
05:09:47 (2548): wrapper: running fit_sed (24 filters.dat observations.dat)"

After the checkpoint restart, Elapsed time as shown in BOINC Manager reset to zero or one  "fit" unit's runtime but percentage completed was correct. This mucked up "Remaining (estimated)" time too. This is all to do with how the wrapper mechanism interacts with BOINC.

When the task's Elapsed time reset I was concerned that the checkpoint hadn't worked but task completed without any trouble. 

All the tasks I have looked at are successfully writing checkpoint files as seen in wrapper_checkpoint.txt file. Because this application uses a wrapper, "CPU time at last checkpoint" does not show anything in BOINC Manager task properties. It will only show something there (one fit duration) if the application has actually restarted from a checkpoint.

At least that's how a restart was shown on this Windows 7 computer. Other operating systems may behave and/or report things differently.
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 12, 2012, 02:13:01 PM
All cool Dingo, gives me a chance to clear my cache/s.
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 12, 2012, 02:32:23 PM
What I meant to imply by my previous post is that due to the way it is reporting some people may think that checkpointing is not working when it is. For example, Dataman's 4 aborted tasks show 3 fits successfully completed and 2 restarts on fit 4, although what was shown to him by BOINC Manager made him think those tasks had restarted from the beginning.

Wait for one fit duration (often about 2% of total task time) then have a look in your stderr.txt file for that task in your BOINC slots folder if you think checkpointing hasn't worked.
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 12, 2012, 02:46:46 PM
Does anyone use BoincTasks? It track's checkpoints in real time & also CPU useage. (may have to be activated in settings?)

It paints a strange picture, checkpoints are always in the red (overtime) & CPU usage shows at between 2 & 15% also in the red? What the Fred does that mean?

Anyway it's all good Dingo just minor teething probs, we'll crunch up a storm when all the small bugs are fixed.. :wink

Cheers ..JN..
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 12, 2012, 03:08:28 PM
If BoincTasks is based on the same information as BOINC Manager, it will have the same reporting limitations with these POGS wrapper tasks. In other words it will not display checkpoints or CPU time correctly, the same as BOINC Manager. With these wrapper tasks on Windows the CPU time resets at the end of each fit, that's why most completed Windows tasks show zero CPU time reported on the POGS website after the concatenation processing. For the same reason, during the task processing the BOINC Manager CPU time in Properties shows the time since the last fit was started, which is also the time since the last checkpoint.
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 12, 2012, 07:36:50 PM

Just turned on my PC and hello hello ALL 9 pogs WUs restarted correctly except for the time remaining figures (as explained in Kashi's earlier post).
checked all the files as described on the forum and they are all there and contain the right information.  I guess I got caught out with the BOINC display as well.

When is the next AA??
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 12, 2012, 10:55:01 PM
The next AA starts on the 1st September.  I have sent a message to Kevin at theSkyNet to see if he thinks it can handle an AA.

Quote

Hi Kevin,

Doing a great job with the project, only complaints at the present from my members is that check pointing is not working on Windows PC's.

Four times a year the Team has an Aussie Assault (AA) where the team picks a project that we get as many Team members as possible to work on at one time. This boosts the Team together and we usually pick a project that we need a boost in the standings on.

The next AA is due in September and I am thinking that I will propose this project as it is the first Australian Project. The team has over 4000 members but only about a 1000 usually join but because the project is Australian there may be more.

Do you think your project can handle the work load and there will be enough work available between 01 - 14 September 2012.

Cheers
Dingo


I just got his reply which looks like it may be ready he does't know, especially if he needs to write his own wrapper.
Quote
I thing I'll have to ditch the wrapper and write my own :-(

I've just got 216 new galaxies - so maybe....

We can test it over the next two weeks

I will put it up as an option and if it is not ready we can do the AA on the alternate.
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 13, 2012, 12:54:07 AM
Excellent news Dingo.  I'll keep my fingers crossed.  :thumbsup:

Like Lawry said and previously by kashi the check-pointing does work for me also.    It just doesn't appear to be working as kashi mentioned earlier.  As usual Kashi's right on the money.

Anyway things are looking up on the theSkyNet front, I wish the WU's were a bit smaller & the stats a bit more regular, but you can't have everything I guess.   And an AA would be a great way to help spread the word & help bolster our score.  Sooo...

Bring on the AA.. :crazy

Cheers to all ..JN..
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 13, 2012, 02:04:55 AM
Ok mates, thanks for the information. I'll try to grab another couple of wu's and watch them. Still little crunching going on here ... hot Hot HOT! (104F)
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 13, 2012, 03:14:53 AM
POGS hates me.  :hbang: I started 8 on a i7 920 and they are running at 22-27% CPU. I set it back to no new tasks but will try to let them complete but it is going to take 4x the time. Maybe they will use more CPU later.  :cry2:
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 13, 2012, 03:53:02 AM
Is it showing only 22-27% total CPU Usage in Windows Task Manager for 8 tasks combined? Very strange, I'm running 7 cores and total CPU Usage is 87% as expected. Maybe running all 8 cores is bogging it down, especially if you are running a GPU project at the same time. Perhaps you could try 7 cores (87.5% of processors) and see if it improves.

You're not using BoincTasks to monitor CPU Usage are you? That will give incorrect CPU usage readings on this wrapper project in Windows as reported by JugNut here and Saenger on the project forum.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 13, 2012, 04:29:33 AM
Quote from: kashi on August 13, 2012, 03:53:02 AM
Is it showing only 22-27% total CPU Usage in Windows Task Manager for 8 tasks combined? Very strange, I'm running 7 cores and total CPU Usage is 87% as expected. Maybe running all 8 cores is bogging it down, especially if you are running a GPU project at the same time. Perhaps you could try 7 cores (87.5% of processors) and see if it improves.

You're not using BoincTasks to monitor CPU Usage are you? That will give incorrect CPU usage readings on this wrapper project in Windows as reported by JugNut here and Saenger on the project forum.
Yes, showing in BOINCTasks that's all I use. I got so use to remote viewing while we on our walkabout I almost never look at individual machines. After all I would have to stand up and walk 30 ft to the server room.  :rofl:
I guess my point is POGS is too much bother right now. I will let these 8 finish though to see what other things pop up. Thanks Kashi.
:cheers:
Title: Re: Testing of BOINC version of theSkyNet
Post by: danhtruong on August 13, 2012, 09:58:21 AM
Good morning everyone,

I'm running the Java version of theSkyNet from their website with a handful of credits.
With the BOINC version, will those credits automatically attach, or is the BOINC version different/separate altogther?  aka., theSkyNet POGS, as opposed to normal theSkyNet.

(The email address I use to register for both BOINC projects and theSkyNet are the same)
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 13, 2012, 10:20:48 AM

Hello Dan

I believe "they" are currently looking at methods to combine credits, badges etc and DO intend to combine the two.  POGS will be moving to the SKYNET server when testing is complete.
There is a post somewhere on the POGS forum pages but I'm damned if I can find it again.

Cheers Mate

Lawry

EDIT:   If my typing was a bit better I would have found this http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=19&postid=80 (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=19&postid=80)
then scroll down to message 80.

Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 13, 2012, 01:12:55 PM
Kevin has implemented his own wrapper for Windows and it now checkpoints.  Iturned my PC off last night when they were 80 odd % complete and this morning when I started it back up they all went back to where they finished.  I put a post HERE  (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=1&postid=202#202)

Kevin has not announced that it is fixed yet but looks like it is ???
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 13, 2012, 02:07:02 PM

Thanks Dingo.
Got to admire Kevin for his work ethic.

Looks like they might be holding back on any new releases until they confirm the data already processed which makes a whole lot of sense.


Cheers
Title: Re: Testing of BOINC version of theSkyNet
Post by: danhtruong on August 13, 2012, 02:35:14 PM
That's great!
Good to know our badges will carry over.

But yes, I'm sure the first Australian BOINC project will be warmly welcomed; most of all, by BOINC@Australia.

:oz:
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 13, 2012, 03:00:17 PM
Well, all finished "normally" whatever that is. About 9 hrs per. I'm thinking eFMer is going to have to make some changes. I saw a lot of strange stuff in BOINCTasks. When I looked at the machine via Task Manager the wu's were using in the high 80's to low 90's cpu. I'm going to see what asteroids@home does tomorrow before releasing any more of POGS. I am only running one machine with 8 cores and two cards ... hot Hot HOT ... getting to be annoying.  :furious:

:aus1: :cheers: :US
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 13, 2012, 03:28:37 PM
Testing Halted for a while whilr Kevin checks the data to see if all goes well with the science.  See this POST (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=50)
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 16, 2012, 05:29:35 PM
New WU's coming through "now".. (just when I have a huge bunch of SIMAP too  :cry2:)


EDIT: 1 machine got 50 the others none, no matter how many times I updated & increased cache. (but it's a start :-) )

EDIT 2: Coming again in dribs & drabs..   "TESTING -  Offically restarted".. :crazy
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 16, 2012, 06:07:44 PM
Yes I have a bunch of BOINC Simap as well.  I have both projects available for work so it will sort itself out.  :dance2:
Title: Re: Testing of BOINC version of theSkyNet
Post by: tazzduke on August 17, 2012, 01:17:54 AM
On it nabbed about 30 tasks, which will do me for the time being, as I still have some LHC to do

Regards
tazzduke
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 21, 2012, 12:26:08 AM
The project will be out of work for a week (or so) while they check the results that have been returned.
:cheers:
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 21, 2012, 07:47:18 AM


Just as well too 'cause I can feel the hot winds of California blowing on my back.   :shock
Title: Re: Testing of BOINC version of theSkyNet
Post by: danhtruong on August 21, 2012, 02:40:49 PM
Hi Dingo,

Just wondering, for your personal BOINCstats (theSkyNet POGS of 17,000 cobblestones or so) was that a direct conversion from theSkyNet java applet or did you with zero credit on theSkyNet POGS?

I'm still running theSkyNet from their website and just wondering if i should start running the BOINC theSkyNet instead (if its the same thing?) or continue to run from their website.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 21, 2012, 03:56:06 PM
I started from Zero on the BOINC project.  When the project is ported over to the University site from Amazon they say that the Badges and points from the non BOINC project should be coming over.
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 22, 2012, 12:08:34 PM

I stand suitably repentant  :faint:

Read a couple of the posts on the POG forum last night about the number of WUs some people (called "suckers") were downloading.  I guess the purpose of an Alpha test is to get stuff back to the developer as soon as possible.  Pretty obvious now, but I didn't give it any thought when I downloaded heaps of WUs from the last batch.   Seems Kevin had to issue shorter WUs so he could get some returns.  I was feeling such a DH for not seeing the bleeding obvious that I aborted my 55 remaining WUs and sent Kevin a PM.
He was gracious enough to suggest that my actions were a good test for the BOINC system.   Amusing thing was that those 55 WUs lasted about 12 seconds in the download queue.

Just something to think about when the next batch get released perhaps.

@ Kashi.   Great response to the post regarding the time it takes to process work units.  Thank you.

http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=65#384 (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=65#384) is well worth a look folks.
Title: Re: Testing of BOINC version of theSkyNet
Post by: tazzduke on August 22, 2012, 08:39:46 PM
Greetings All

Dingo or other Senior Member could we maybe suggest to Kevin over at POGS of a limit of workunits per core like they do at LHC@Home and other projects, that way he can expect quick turnaround of workunits, this would also alleviate the sucker problem, I would go at say 2 or 4 per core, in my case a max of 16 workunits at one time. 

This maybe lifted once into either Beta or Production, by that time more crunchers have jumped onboard.  I find it disheartening when I come across a cruncher who has amassed over a 100 workunits and take anywhere up to a week to process.  In other words very long time in PV jail.

Any thoughts???

Regards
Tazzduke

Crunching for the benefit of Humantiy  v:
Title: Re: Testing of BOINC version of theSkyNet
Post by: Mike Mitchell on August 22, 2012, 10:58:17 PM
Quote from: tazzduke on August 22, 2012, 08:39:46 PM
I find it disheartening when I come across a cruncher who has amassed over a 100 workunits and take anywhere up to a week to process.  In other words very long time in PV jail.

My pet peeve are the people who download a hundred or so and then do nothing with those work units (in any project).  :furious: Like the idea Tazz
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 23, 2012, 02:29:02 AM
Re my post at the POGS forum. Most Intel computers running Windows will complete each step much faster than Sajjad Imam's AMD FX-8120. I have looked at one of his recently completed tasks and something seems awry. It took over 20 hours for a task that would complete in about 5 hours on my computer. I know Bulldozer architecture is inefficient on some applications due to the shared floating point core per module but that difference seems too high. His times of 35-40 minutes or longer per step are very slow and not consistent. This indicates severe contention losses and/or possible CPU overcommitment. Some people get their back up if I suggest using less BOINC cores in an attempt to reduce contention losses, so I rarely mention it now but I will make an exception in this case. If my suggestion is not appreciated at least I have tried to help.

If Kevin is keen to get the results back quicker then an option is to reduce the deadline from the present 7 days to 4 or 5 days. One problem with limiting the number of tasks per core is the variability of task lengths. If you get a batch of tasks with only a small number of steps (fits, pixels) then these complete very quickly on a fast computer. It means if the project has a temporary server outage or break in work availability then you are out of work. Same if your own internet connection is unreliable. I would not recommend a deadline of less than 3 days unless the project really needs it, because that can interfere with sharing with other projects due to tasks going into high priority mode. It may also cause problems for those with slower computers who download a batch of longer tasks. 

Restricting the number of tasks does address extreme cache hogs to some extent, but it still only partially solves the problem of new members who download a full batch of tasks and complete only a few or none at all and do not abort unfinished tasks but just let them all time out. These anti-abortion hogs cause a longer delay than those with a large cache but a turnaround time of 2-3 days. With a restriction of 4 tasks per core, someone with an 8 core computer who downloads a batch of tasks then only completes 2, ties up 30 tasks for the full 7 days and then the potential 7 day cycle starts again when the tasks time out and are reissued. When batches are large,  sometimes it can take longer than 7 days for a task to be reissued because it can remain in unsent state for some time.

Not everyone reads a project forum and realises that a project wishes a quick turnaround time. If the deadline is 7 days then that's how long some will take to finish tasks.

When a batch of work needs to be checked before more work is released then a shorter deadline is a good way to reduce the total turnaround time of first issue and timeout reissue tasks.

If the project has an astronomical amount of data to process then attracting and keeping a decent size number of contributors is vital. I understand that the project is only new and batches need to be checked at this early stage so interruptions to work availability are currently unavoidable. However now that the project is open to all it is important that work is reliably and consistently available as soon as possible. Lack of work causes contributors to switch to other projects and if "no work available" is the introduction to a project any new contributors receive then some of those may not return. Lack of consistent work availability may also disqualify a project from being considered by teams as their special project for a time (like our Aussie Assaults).
Title: Re: Testing of BOINC version of theSkyNet
Post by: tazzduke on August 23, 2012, 07:24:26 AM
Greetings Kashi

Thankyou for your input, has given me a new understanding.

Regards
Tazzduke
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 23, 2012, 07:46:50 AM
@Kashi

Thanks Kashi.

I noted your earlier post (to dataman) about reducing the number of cores crunching, tried it, and yes it was much more efficient and also had a positive effect on my GPU computing.  
There is a post on Poem (I think) about the efficiency of GPU/CPU use (I will try to find it again and post a link here).

I think I would prefer a reduced deadline and/or smaller WUs (for the testing period) rather than a limit in downloads though.

Early days yet though and Kevin seems to be heading in the right direction.

Mate, for every one person who gets upset about suggestions there are 10 more who read AND digest.  Please keep it up.

Cheers

Lawry

EDIT:  http://boinc.fzk.de/poem/forum_thread.php?id=706 (http://boinc.fzk.de/poem/forum_thread.php?id=706)  The Poem link.

Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 23, 2012, 09:37:38 AM
Ah yes, but POEM GPU is an extreme case. It needs more CPU than any other released GPU project. So much so in fact that if you wanted to do the maximum amount of work and were not concerned with also supporting some CPU projects, then the greatest efficiency is achieved by not running any CPU projects at all! This is rather revolutionary for BOINC in a sense as previously GPU projects were considered as an addition to CPU projects on multi core computers not as a stand-alone crunching concept.

As you are aware there are some GPU applications where CPU usage is very small, mainly the ATI/AMD ones based on CAL/Stream originally optimised with IL code by Andreas Przystawik such as Collatz and MilkyWay.

Even though one of the POEM GPU developers states: "our application can not run on GPUs only, but also needs CPU calculations between GPU steps. This is a major difference to other GPU projects hosted by BOINC, and causes the non-optimal usage of GPU and CPU by bandwidth limitations", I still believe the POEM application was developed on older previous generation GPU hardware and could be improved to better utilise recent more powerful GPUs by modifying a combination of sleep time, chunk size, kernel calls, etc. so as to reduce CPU throttling thereby increasing GPU load percentage and increasing efficiency. User adjustable parameters to enable fine tuning the GPU application to suit different hardware and user needs would be great but unfortunately are very unlikely to be provided. It appears as though it took them so long to develop the application that they are now reluctant or have insufficient resources to even attempt improving or refining it.
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 23, 2012, 10:31:29 AM
My god Kashi every time I think i've made one step forward about understanding parts of Boinc I read one of your posts & realize i've not scratched the bloody surface.

I despise butt kissers but credit where credits due my friend "you rock"

Maybe you could suggest a "Boinc for dummies book" for the thick as a brick type such as me.

Bashhead
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 23, 2012, 01:18:07 PM
Kashi -  Thank you for the explanations for this topic and for the many you have given in the past.  You are a valuable member of the team and able to explain things much better than myself.  I may know what you are saying but cannot say it in such an elegant and straight forward way as you do.  +1 for your participation in the Team and the forum. :worship
Title: Re: Testing of BOINC version of theSkyNet
Post by: kashi on August 23, 2012, 03:25:03 PM
Aw shucks, thank you both. :blush1:

There are many who know a lot about BOINC but do not have either the time or the inclination to explain it in detail. I have taken a great interest in the development of GPU applications and have been privileged to be able to help some of the developers by testing and reporting on development versions of GPU application versions. The detailed workings of the BOINC program code itself still baffle me often however even after all this time. biggrin

I have not been able to spend much time on the forum in the last few months due to my mother's illness. She has been in various hospitals for the last ten weeks since an operation to repair a massive hiatus hernia in June. Will be off to visit her again in an hour at Berkeley Vale which is about 30-40 minutes north of here. She has improved greatly in the last 2 weeks and should be coming home in about 10 days.

Had a mini drama here yesterday, when I got up I could hear the water running strongly and thought my brother was in the shower. So I'm eating my breakfast thinking, "the bastard has been in there a long time, he'll use all the hot water again." Meanwhile he was out on the back verandah playing with next door's cat and had been thinking the same thing about me. Turns out we had a burst water pipe connection along the side of the house. The plumber was cursing when he discovered someone had used a section of inferior grade white plastic pipe, instead of some better quality black plastic or copper pipe. He is based a long way away from here and did not have the special connector needed in his truck so went off to a local supply company to get one. They did not have one as they are rare now, so he had to put in a new section of copper pipe. We were thinking it was going to cost a fortune since it all took about 3 hours but he only charged $200. So all's well that ends well.  :thumbsup:
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 23, 2012, 04:13:35 PM

Great news your Mum has improved and will be home soon Kashi.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 23, 2012, 04:49:01 PM
Hope all goes well with you mother and she gets well soon.  Mothers are very special people and I miss mine very much since she passed two years ago.

I to had a broken water pipe in my backyard a couple of weeks ago.  The plumber was to come back and fill in the hole when it dried out.  Although I have called him a couple of time he has not shown up to fill it in.  I am not complaining too much as he has not sent a bill either.  If the bill arrives and the hole is not filled in I will call him again.   I do not have any shovels or I would fill it in but I think I will wait a while for the plumber.
Title: Re: Testing of BOINC version of theSkyNet
Post by: tazzduke on August 24, 2012, 01:05:25 AM
Greetings Kashi,

This is a carry on from my previous post, as now I dont have two little ones screaming for my attention.

+1 for your responses, I am still in early days on the learning curve with BOINC and its charms. 

Am glad to see that when the questions are put out to the members, we get answers that come across from different perspectives which makes this forum rock.

Wishing your mum all the best on her road to recovery

Regards
Tazzduke
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 24, 2012, 01:51:19 AM
As the others have mentioned, I also wish your mum a full & speedy recovery mate.

All the best ..JN..
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 29, 2012, 08:01:40 PM
Work units available.  Kevin has limited downloads to 10 WUs per core.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Mike Mitchell on August 29, 2012, 09:58:53 PM
Thanks Lawry, I grabbed a few. Plus one for the heads-up.   :thanks1:
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 29, 2012, 11:59:16 PM
Yea thanks Lawry,  :thumbsup:

I wish I would have seen your post a tad earlier though.   Oh well such is life.    +1 Karma for your sharp eye.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 30, 2012, 05:50:16 AM
I put 76 threads there. That should help their stress test (and bump up my numbers  :jester: ) So far so good ... no errors and only 1 inconclusive.
:aus1:
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on August 30, 2012, 06:55:12 AM
Way to get stuck into it DM   :thumbsup:

Oh and try and leave one or two for the rest of us.. "Please"  biggrin
Title: Re: Testing of BOINC version of theSkyNet
Post by: tazzduke on August 30, 2012, 07:03:09 AM
Way to go Dataman, I will have to wait till the end of AA31, as I have loaded up with YOYO tasks.

Regards
Tazzduke
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on August 30, 2012, 11:27:00 AM
I am doing a few tasks before the AA starts and all is good so far. 
Title: Re: Testing of BOINC version of theSkyNet
Post by: LawryB on August 30, 2012, 04:09:19 PM

@Dataman.  Only 76 threads?  You not trying or something? Bloody hell that is only 70 threads more than me.  Go man Go!
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dataman on August 31, 2012, 12:17:14 AM
Quote from: LawryB on August 30, 2012, 04:09:19 PM

@Dataman.  Only 76 threads?  You not trying or something? Bloody hell that is only 70 threads more than me.  Go man Go!
I have a maximum of 80 but I can on run half of them on this 8 core machine or it screws up my video security system. I had to shut down some yesterday as it is just too hot here. Was 97F yesterday and 103 in the server room. I have 40 running now but will probably have to shut down around noon. It will be 97F again.
 :AUS: :cheers:  :USA1


EDIT:  I changed the flags for the smiley so needed to change here
Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on September 15, 2012, 07:41:45 PM
Anyone know whats happened to the stats export?      Almost 37hrs since last update on Boinc Stats.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on September 15, 2012, 09:58:50 PM
There's a post  (http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/forum_thread.php?id=44&postid=563#563) on their Web Site but no answer yet.

Title: Re: Testing of BOINC version of theSkyNet
Post by: JugNut on September 15, 2012, 10:47:48 PM
Thanks mate!  There was nothing before.   As long as they know then all's good.
Title: Re: Testing of BOINC version of theSkyNet
Post by: Dingo on October 06, 2012, 05:58:30 PM
Looks like the new client is available and work as well.  There might be a problem, I am still downloading new work.

Quote

News

Warning
The new version needs new models, unfortunately the BOINC client will start trying to run things before it has finished downloading them.

It'll either go really quickly or fail with computation errors. I'll register the early galaxies again so they get processed. 6 Oct 2012 | 4:09:21 UTC ยท Comment
New galaxies
Folks

We're ready to go with the new galaxies, we're just waiting for the last few results of the old run to come in. We've made a number of improvements:


1) The F77 records more probability data than before;
2) We've decided to push the redshift out to 0.1;
3) The next run is pure PanSTARRS1 data;
4) The 64bit Windows code should be a lot faster;
5) A improvement to the Cobblestone scaling to look at the number of layers you are processing as well as the number of pixels.


WARNING: The model files have grown in size so you'll get a once of down load of 20MB (gzipped) 5 Oct 2012 | 9:30:42 UTC