Multiple LINUX BOINC Instances

[Dingo]

As users are creating these manually for nci projects to run in I thought I would try and create some scripts that do it in bulk.   This would have taken a programmer or someone that knew Linux and Bash about an hour to work out. 

I have been trying to get this working since 9AM this morning and it is now just after 9PM and I eventually got it to work.

It works on "Ubuntu 18.04.3 LTS" but should work on most Linux types, hopefully.

I have 150 instances nci.goofy and wuProp on my Linux box BUNDY-3

OK this is the process.  It looks complicated but not too bad.

The syntax is very important so if you chnage anything it may not work.
I was logged in as root so if you are not then you have to add sudo to all the commands in the terminal.  Not the ones in the scripts.


Create Multiple BOINC instances  LINUX


If you have a GUI the Ctrl+Alt+t will open a terminal window:

mkdir /usr/bin/multiboinc

mkdir /usr/bin/multiboinc/BOINC2

cd /usr/bin

**Note Copy the files for the projects that you want to run in the instances. Must have installed these on the machine in the default location for this to work.

cp account_nci.goofyxgridathome.net.xml multiboinc/BOINC2

cp account_root.ithena.net_usr.xml multiboinc/BOINC2

cd multiboinc

I use nano but use the editor you know.

nano create_folders.sh
insert the following

#!/usr/bin/bash

n=100;
max=110;
while [ "$n" -le "$max" ]; do
  mkdir "BOINC_$n"
  n=`expr "$n" + 1`;
done

save the files created by nano by pressing Ctrl then y if it OK press y then press enter.

This will create 10 folders starting at n BOINC_100 and finishing at max BOINC_110
sh create_folders.sh 

nano copy-files.sh

Insert the following

#!/usr/bin/bash

n=100;
max=110;
while [ "$n" -le "$max" ]; do
  cp BOINC2/* "BOINC_$n"
  n=`expr "$n" + 1`;
done


This will copy the contents of BOINC2 to the folders created above.

sh copy-files.sh

This will start the BOINC instances from n to max

nano start_instances.sh

#!/usr/bin/bash
n=100;
max=300;
while [ "$n" -le "$max" ]; do
  /usr/bin/boinc --daemon --allow_multiple_clients --dir "/usr/bin/multiboinc/BOINC_$n" --gui_rpc_port "50$n"
  n=`expr "$n" + 1`;
echo "$n "PORT:"50$n"
done

sh start_instances.sh

Other command that you can do from the command line

Stop all boinc instances:  It takes a minute for them all to stop

/etc/init.d/boinc-client stop

Check on the status of BOINC

/etc/init.d/boinc-client status

If for some reason boinc-client does not stop boinc then create a file called /etc/init.d/boinc

Code Select Expand


nano /etc/init.d/boinc

Insert into the file:

#!/bin/bash
# BOINC start/stop
#
if [ $1 == start ];then
cd /usr/bin
./boinc --daemon --allow_remote_gui_rpc
fi
if [ $1 == stop ];then
   pkill boinc
fi

to stop boinc

/etc/init/boinc stop


It takes a while for them to stop. 

If you have any problems it will probably be fat fingers as that is my main problem.

EDIT: 06 Jan 2020

If you have Linux with GUI then in a command window Ctrl+Alt+t if you enter this it will open that instance in Boinc Manager:

/usr/bin/boincmgr -m -n 127.0.0.1 -g 50101 -p password

EDIT: 26 Jan - Added change password script:

This is the change password script for Linux based on the mult BOINC instances installation that I started this post with.
If you created your instances using the instructions in this post then the gui_rpc_auth.cfg file contains a 32 character password.  If you want to know the password of an instance then you need to look it up in that file and every instance is different.  Setting all the passwords the same should make it easier to attach to BoincTasks and run scripts.

I still haven't been able to attach to an Instance of BOINC with a script.  I always get the error "can't connect to 127.0.1.1"

format to run this acript is sh change_password.sh -a your password -b anything but must enter something (Not used)

nano change_password.sh

Insert this code and change the n= to the starting instance and max= to the last instance.  Try it out on a couple and have a look and make sure it worked before doing all instances.

press Ctrl+x then y then enter to save

Code Select Expand



#!/usr/bin/bash
#!/bin/bash
#!/bin/bash
#
#  Script Name change_password.sh
#
#
helpFunction()
{
   echo ""
   echo "Eg: sh change_password.sh -a New Password -b Press any key as not used."
   echo "Usage: $0 -a"
   echo -e "\t-a Enter the new password that will go on all instances in the loop"
   echo -e "\t-b Not used but must enter something"
       exit 1 # Exit script after printing help
}

while getopts "a:b:" opt
do
   case "$opt" in
      a ) passwd="$OPTARG" ;;
      b ) operation="$OPTARG" ;;

      ? ) helpFunction ;; # Print helpFunction in case parameter is non-existent
   esac
done

# Print helpFunction in case parameters are empty
if [ -z "$passwd" ] || [ -z "$operation" ]
then
   echo "Some or all of the parameters are empty";
   helpFunction
fi
n=100;
max=102;
while [ "$n" -le "$max" ]; do

   echo "$passwd"> /usr/bin/multiboinc/BOINC_"$n"/gui_rpc_auth.cfg;
   echo "BOINC_$n $passwd";

   n=`expr "$n" + 1`;
done



[tazzduke]

Greetings Dingo

Thankyou for your persistence and solution.

I aim to give it a try before XMAS.

I tried out the Windows one (website metioned in another thread) but I want it on a Linux box that just sits and does Boinc only.

Cheers

[JugNut]

@ Dingo: You'll probably find 99% of your WUprop work units will end as invalid and give no credit. WUprop had a major dummy spit about using multiple boinc instances some time ago and now refuses to count goofyXgrid hours or even validate the WUprop work unit "if" there is only goofyXgrid work units running by themselves.

[NudgeyNR]

This is the commands i use. Notice i have added "-daemon", this allows me to run multiple copies without keeping a terminal open. They remain running (hidden) and viewable via BoincTasks.

Code Select Expand

#!/bin/bash

echo "Starting MultiBoincs 100-160"

for x in {100..160}; do
   /usr/bin/boinc -daemon -allow_multiple_clients -dir BOINC$x -gui_rpc_port 50$x;
done

echo "Commands executed..."

I followed this site for windows and worked out how to do the commands in linux.

Ie:

Code Select Expand

mkdir BOINC{100..160} to make multiple directories BOINC100-BOINC160.

Fantastic to see how you copied the files to the multiple directories. Couldn't work out the command, so did it manually via GUI.

[Dingo]

@ Dingo: You'll probably find 99% of your WUprop work units will end as invalid and give no credit. WUprop had a major dummy spit about using multiple boinc instances some time ago and now refuses to count goofyXgrid hours or even validate the WUprop work unit "if" there is only goofyXgrid work units running by themselves.

@JugNutThanks for the info.  I will have to have a look tomorrow and work out how to detach from wuprop on all the instances.  Shouldn't be too hard.  Isn't there another nci project that started rcently ??  Can't remember the name off the top of my head will look tomorrow.

@NudgeyNR - I had problems with the {100..160}; part and couldn't get it to work.  But I didn't have the semicolon after {100..160} and that must have been the problem.  There is more than one way to skin a cat as they say, or used to say.

[Dingo]

OK I spent a couple of hours creating a script to first send nomorwork and then another to detach from the project but with both I run into the same problem.  With boinccmd I get the error "can't connect to local host" which there are lots of pages on google about it but I could not fix the problem, even running it in the BOINC_100 folder where the gui_rpc_ath.cfg is located.

Even a simple command like this with the correct data entered gets the error

Code Select Expand

boinccmd --lookup_account www.myproject.org myemail@email.com mypassword


If anyone finds a way to do a boinccmd command from the command line please let me know how you did it.

[JugNut]

boinccmd.exe --host 127.0.0.1:yourportnumber --passwd yourpassword --project http://wuprop.boinc-af.org/ detach

The above is the correct syntax which should work on windows & linux.  I used the command to detach WUprop from three different instances.  While it works fine its not automated so sadly each time you run the command you need to change the port # to match the port # of the instance your trying to remove WUprop from. I did have an automated script but it's gone MIA and this is all I can remember of it now.   If you end up automating it yourself then please post the necessary commands or scripts. 

Below is the exact commands I used. The only difference in the commands is the port numbers used to access the different instances.  The port number below are the ones that I used.  So use your port numbers instead.

boinccmd.exe --host 127.0.0.1:31514 --passwd 99999 --project http://wuprop.boinc-af.org/ detach
boinccmd.exe --host 127.0.0.1:31515 --passwd 99999 --project http://wuprop.boinc-af.org/ detach
boinccmd.exe --host 127.0.0.1:31516 --passwd 99999 --project http://wuprop.boinc-af.org/ detach

PS I'm not sure if this will matter or not but... if the commands don't work make sure your localhost loopback address is the same as mine 127.0.0.1
While 127.0.0.1 is used most commonly it is possible yours might be slightly different. For example my raspberry Pi's use 127.0.1.1 instead.
Maybe that's a Raspberry Pi thing or maybe it's a debian thing but it's worth checking nonetheless. If yours is different than the 127.0.0.1 that I used then try using your loopback number instead.  In the end it's exactly the same thing but it might make a difference to the command working or not on your particular system.  It might make a difference or it might not.  IF you have something different then just replace 127.0.0.1 with 127.0.1.1 or whatever yours is and try again. 

Hope this helps.

[Dingo]

I will give it a try tomorrow, I was on my computer most of the day again, that is two days in a row.  Keeps me busy and the brain active I guess.

OK I couldn't wait I gave it a try and I still get the can't connect error.

Code Select Expand



hostname -i
127.0.1.1


/usr/bin/boinccmd --host 127.0.1.1:50100 --passwd 6e6bb01c384ff11f60d33763b5c1517e --project https://root.ithena.net/usr/ detach
can't connect to 127.0.1.1

/usr/bin/boinccmd --host 127.0.1.1:50100 --passwd 6e6bb01c384ff11f60d33763b5c1517e --project https://root.ithena.net/usr/ nomorework
can't connect to 127.0.1.1



[JugNut]

Silly question Dingo, but if the instances are viewable within boinc-tasks can't you just select the ones you want then just detach the normal way?

Hmm also it seems our badge signatures have gone missing.  Hope it's not a long term thing.

[Dingo]

Silly question Dingo, but if the instances are viewable within boinc-tasks can't you just select the ones you want then just detach the normal way?

Hmm also it seems our badge signatures have gone missing.  Hope it's not a long term thing.

I don't have them attached to boinctasks, only stop and start on the command line.  I have 40 instances on this Linux box, but on my main Windows machine I have 150 so will probably increase the linux one with more instances.  I should have gave them all the same password in the script which I will change today or tomorrow so that I could attach them eisier.  When I search for computers on BoincTasks, it is on the Windows machine, it does come up with 40 comoputers but I can't attach as they all have the 32 character password and that would be a pain.

I will keep trying, going to try it on Windows and it will probaly work fine there.  I think the Linux problem might be something to do with permissions, I will have a look, so maybe another half a day this time on the computer.

[JugNut]

Yea I thought it might be something like that. Yea your right, it may not be as secure but it's absolutely necessary to have your boinc passwords all the same. Or at the very least all instances on the same PC should have the same password. As you say if you would have changed the passwords all would be now connected through boinc-tasks and you'd be able to detach with ease.  That's also why the above commands aren't working either as each needs the correct password.  But at least you know what the problem is and how to fix it.

[Dingo]

I now realise why when I tried to start 40 instances of ithene that it went to 100% CPU and locked up the machine.

I used the wrong url and actually added boinc.tbrada.eu which is not a nci application.

Old timers is kicking in a lot recently when I am tired. 

[Dingo]

I have been working on scripts so that I can detach and set no more work etc but have had a problem of accessing the file as I keep getting an error.  It had bugged the be Jesus out of me.

I gave up on those scripts for a while but i wrote a script that changes the password held in the gui_rpc_auth.cfg.

The first one here is for Ubuntu Linux on WIN10 that changes the Windows 10 installation of multiple BOINC instances and will only work because of the folder names etc if you used these instructions on WIN10

but tomorrow I should be able to modify it to work on Linux.

the format for running the script is

sh change_password -a new password -b anything not used in this script

nano change_password.sh

insert this and remember to change the parameters "n" which is the first machine to change and "max" which is the last machine to change before you run th script

Code Select Expand


#!/usr/bin/bash
#!/bin/bash
#!/bin/bash
#
#  Script Name change_passwd.sh
#
#
helpFunction()
{
   echo ""
   echo "Eg: sh change_passwd.sh -a New Password -b Press any key as not used."
   echo "Usage: $0 -a"
   echo -e "\t-a Enter the new password that will go on all instances in the loop"
   echo -e "\t-b Not used but must enter something"
       exit 1 # Exit script after printing help
}

while getopts "a:b:" opt
do
   case "$opt" in
      a ) passwd="$OPTARG" ;;
      b ) operation="$OPTARG" ;;

      ? ) helpFunction ;; # Print helpFunction in case parameter is non-existent
   esac
done

# Print helpFunction in case parameters are empty
if [ -z "$passwd" ] || [ -z "$operation" ]
then
   echo "Some or all of the parameters are empty";
   helpFunction
fi

n=100;
max=102;
while [ "$n" -le "$max" ]; do
(
    break> /mnt/c/multiboinc/BOINC_"$n"/gui_rpc_auth.cfg;
)
   echo "$passwd"> /mnt/c/multiboinc/BOINC_"$n"/gui_rpc_auth.cfg;
echo BOINC_"$n";

   n=`expr "$n" + 1`;
done


If you do not have Linux installed on WIN10 and want to give it a try I think these are the instructions I used Installing Linux on WIN 10

[jave808]

@Dingo, this may seem like a silly question, but what are the advantages of running multiple BOINC instances?

[Dingo]

@jave808:  There are never any silly questions.  The only advantage I use it for is running Non CPU Intensive (NCI) tasks like http://nci.goofyxgridathome.net/.

Because they do not use much CPU, about 0.025 of a thread, you can have a lot of instances running it without affecting BOINC or your computer, although I did reduce the number of cores available for the main BOINC instance.

Others use it to Bunker tasks but that is just too difficult for me, and I don't bunker.