Re: Project Update

[chooka03]

Never knew what Free DC did.
Thx 

[chooka03]

CUDA app testing.

Are we about to see an explosion of credit points with this project for people with NVIDIA cards? Looks like it.

https://boinc.drugdiscoveryathome.com

[tazzduke]

Greetings All

It does have some special requirements, which is explained via the link that chooka03 has posted.

Out of my 4 crunchers, only 1 can do it and thats the AMD Ryzen 1400 due to it having the required AVX2 cpu component.

The others are Core2Quad crunchers.

[kashi]

Seems like it's missing a very basic "special requirement" to work on my Skylake computer also. "No tasks are available for gromacs."

Forget if this project uses CreditNew or not, for CPU it is known to be an unreliable and inconsistent system, but using it for GPU tasks can result in very extreme swings in credit rate.

I'll give it a go if there are any more tasks issued.
Thanks for the alert Chooka.

[JugNut]

These WU's don't require AVX2 but as I found out they do need AVX in order to run.  The WU's themselves are extremely poor on GPU utilisation, on my bigger cards 1 WU's uses just 7% - 8% of the GPU load.  In fact even when running multiple(up to five) using an app_config I couldn't get past 75% load. But unlike all other GPU apps i've ever used running multiple instances at once doesn't increase the runtime of any single one of them.  Stranger still even at only 75% load most of my cards heat up just like they were under full load,  the same as running 2 of einsteins GPU WU's in parallel would do.

While the credits are OK i'm still not a fan of these as yet as there just seems to be to many unknown questions about how and why they run the way they do.
Also i'm pretty sure the GPU WU's are not using credit new but still credits are all over the place.  When I first looked at what others were getting I thought wow that's alright 250k-450k per 6 - 8hr work unit but the longest WU i've run so far gave 72,208 credit. But even then I have  quite a few other WU's that ran very similar times but only gave 25,000 credit. It seems to be a lucky dip,  if you're lucky you could clean big time.  If not then you "should" still do OK.  Although it seems at least possible to do poorly. In other words things have a looong way before they settle down.  I may finish up the WU's I have then come back an try again at a latter date.

But I suppose thats the price you pay for alpha testing a new GPU app.

For any one interested this is my working app_config that I am now using on a box with 2 x GTX 970,s.  As I need as many free CPU cores as possible for WCG i'm only running 3 WU's in parallel in the app_config listed below(0.33). Just change the numbers as per your needs.
You can remove the vina part of the app_config if you don't need control over the vina CPU app.

<app_config>

<app>
<name>gmx</name>
<gpu_versions>
<gpu_usage>0.33</gpu_usage>
<cpu_usage>0.5</cpu_usage>
<max_concurrent>6</max_concurrent>
</gpu_versions>
</app>

<app>
<name>vina</name>
<max_concurrent>10</max_concurrent>
</app>

</app_config>

[kashi]

I'm doing 2 concurrent on my GTX 970, GPU load is fluctuating between 60% to 72%. The 2 I completed so far paid quite poorly for GPU tasks, one ran 4.7 hours for 16,448 credits, the other 5 hours for 17,400 credits. No 500,000 and higher credit tasks for me, like some others are getting.

Can't think of a reason credit is so erratic unless CreditNew is being used. Unless perhaps the admin is mucking around changing the credit rate with little idea of what he's doing.

A resend with a shorter deadline caused 1 task that was almost finished to stop processing at 85% complete, grrr. When I paused that resend the original task started again from the start. Checkpoints aren't working on my computer even if the admin says they are.

So yes, quite flaky at present, some of the old resends that I let run for a while were showing that they would take over 30 hours to complete. So I aborted those ones. Think I'll give it a miss too after these 2 current tasks complete. They're going to run about 9 hours. If I'm going to risk doing GPU tasks that long I'd prefer more reliable GPUGrid tasks, plus I'd get a more generous and consistent credit rate there as well.

[JugNut]

I'm doing 2 concurrent on my GTX 970, GPU load......

A resend with a shorter deadline caused 1 task that was almost finished to stop processing at 85% complete, grrr.

Yep same happened to me but I was even luckier and was at 99.325%.  When I noticed what had happened I tried suspending the interloper & allowing the the old WU at 99.3% to run again, but then I seen the same thing as you did.  The old WU that was at 99% had restarted from scratch, so then aborted both them.  It was only then did I read that they are supposed to checkpoint in some "other" non-boinc way. But of course by then it was way to late to test out that theory.  But I did check the stderr file before I aborted it & it said there were zero checkpoints.  So in the end i'm buggered if I know? 

It seems strange things be afoot tonight hey Watson?  Sheesh! Where's Holmes when you really need him? 

[kashi]

Yes, I looked at the stderr.txt file in the slots folder, the Gromacs application itself seemed to set a backup checkpoint*, but as far as I could tell the GPU wrapper didn't use it and the task started again from the beginning.

* Stderr.txt said something like: "Back Off! I just backed up md.log to ./#md.log.2#"

My last 2 tasks that were heading for 9 hours just completed at about 5 hours, so the Remaining (estimated) in BOINC Manager is of little use. Similar low credit to my first 2 completed.

Server out of tasks again, I'll leave it requesting work and have one more try to snag a credit whopper but looks like they're finished now.

[chooka03]

Been reading a bit lately about Kaspersky..... maybe these guys work for the FSB looking to use network computing to do their dirty work? 

{Did I just say that!!} 

http://www.skynews.com.au/news/world/nthamerica/2017/10/12/israel-warned-us-of-russia-s-kaspersky-hack.html

[JugNut]

Just finished some version 0.24 WU's that are giving a lot better credit. LINK

I just noticed there is now an even later version 0.25. I'm crossing my fingers that this time they've finally got their credit system sorted and it stays as it is now?  But until I finish some I can't say.  As mentioned you can run 3, 4, 5 or more at time so if credit stays the way it is now it will indeed be a well crediting project. 
For anyone interested in giving them a try there is now plenty of work available.  For how long is anyone's guess?

Crunch-on

EDIT version 0.26 just released,  apparently checkpoints are now showing in boinc & the progress bar is now more accurate.(well it updates progress every 15mins at least)  I have quite a big queue of the other versions so it will be a fair a while until I get to test any of these.  If you crunch any let us know how they go. 

Oh, and what credit they gave too.  LOL can't forget that.

[kashi]

And now there's version 0.26, with checkpoints supposedly working.

Hmm, those tasty v0.24 ones you completed were all old resends originally issued on the 17th. In the past, those were the ones that could sometimes give the more generous credit.

Although I snagged a single v0.24 that was a fresh resend that credited 95,906 for a 2.75 hour runtime. So that's much better than any of my previous tasks. Was using nt 3 parameter too, didn't seem to increase GPU load percentage at all but made the Afterburner graph of it much flatter and less sawtoothed.

Currently my 970 is working on 2 concurrent v0.25 tasks. They're now at about 50% complete after 2 hours, so should finish just before "breakfast". So then we'll see what they grant and if credit system is still wildly inconsistent. Be nice if it stayed generous at least for a while as nt 3 uses up my CPU cores, so WCG badge hunting is slowed a bit.

Aha, you've updated your post while I was writing this post.

Edit: And I just increased my cache and both tasks swapped out and went into Waiting to run state. Each with 2.5 hours runtime and 61% completion lost. Bah!

[JugNut]

Not to worry kashi at least now you get to test out the checkpointing ;)



EDIT: Mmm I Spoke to soon it looks i'll soon get that opportunity as well.  You really have to keep your eye out for those short deadlined resends.  Now if I see one that hasn't yet jumped to lightspeed I abort it straight away.

Oh and what about those uploads & download?  There huge!!  With all the ancillary files the uploads get up around the 36 - 39MB each and downloads aren't far off it either. Lucky most are long runners or it would be a big hit to those without unlimited internet plans.

[kashi]

Haha, yes resume from checkpoint still not working in v0.25, that's why I exclaimed "Bah!". Have set No new tasks and reduced cache setting to hopefully avoid further annoying swap outs. Haven't tried checkpointing on v 0.26, still have oodles of v0.25 left, the first 2  I completed paid exactly the same credit rate as the v0.24 I completed.

Credit scales exactly with runtime for the last 3 tasks completed, so I've reverted to nt 1 from nt 3 and now doing 4 concurrent nt 1 rather than X2 nt 3. Tasks take more than twice as long without nt 3, so X4 nt 1 probably about 10% less efficient, but total credit yield should more than double due to runtime scaling.

GPU load is flicking between 68% and 76%, could possibly go 5 concurrent for a little extra but 4 will hopefully do nicely enough. If credit rate scaling remains the same as last 3 tasks, daily total will be huge at 4 concurrent. 

[JugNut]

Yep thats what I ended up with 4 x concurrent on most my cards except for a single 980 ti that i'm testing 5 at a time on.  If I had 2 980's that would be 10 cores i'd have to keep free.  I'm thinking about stopping CPU work altogether as these guys seem to like as much CPU time as possible. At the moment nt1 is working out nicely so i'm leaving it as is for now.

Also i've noticed on the 5960x the CPU is at maximum power draw something i've never seen before. For some reason they are giving my PC's a good workout as the fans on my boxes are screaming like an army of angry banshee's.



EDIT: Jeeper's!!! If you thought those last WU's credited well then have a look at these... Link..

At this rate i'll have a billion credit before the days out.   
Ok that's a slight exaggeration on my part, but boy they are crediting well to say the least.


EDIT2: With the extra push of that last batch of big crediting jobs I jumped into fourth place...    

Sing it loud..... happy days are here again the skies above are clear again,  let us sing a song of cheer again,  happy days are here again da da da da da dar....(it's an old war song my dad used to sing all the time) Song Link...

EDIT3: Going up...  now in 3rd place..

[kashi]

Yep, you've really hit the jackpot on that machine.  

It's gotta be some form of CreditNew hasn't it, surely nothing else is that wildly, erraticly, inconsistent as to give hugely different credit rates to different computers with similar hardware specs for the same batch of tasks. Don't even need one of the fastest cards either, Tazzduke had a 622K task on his GTX 960 a few days ago.

My last 4 tasks completed and credited at the same hourly rate as the 3 before them. So if I multiply that by 4 concurrent, then by 24 hours, it should yield around 3.3 million, woohoo.

Know that song very well, my mum used to sing it sometimes, she sang it in a concert at Sydney Girls' High about 1941, which she won. Don't remember hearing that version you posted before, can remember a female version from when I was young, probably Patti Page. Then later on one by Max Bygraves. Slow version was Barbra Streisand's "signature song" during the early part of her career and was on her first LP.

Yeh, it's fun roaring up the charts, ain't it. I've zoomed up 130 places in the last 3 days, haha.

Edit: Ah, didn't notice, statseb just informed me of new badge. Oooh, giant, blue and shiny.