DropSpot: Could not contact the given server (error 200) (error 200)

Hi JM,
the output of the Test Connection in Admin Tool is:
_____________
Output will appear here....

Trying to connect to supplied IPAdresses: [192.168.0.43, 192.168.0.41, 192.168.0.42]
/192.168.0.43:667 : 3ms
/192.168.0.43:666 : 3ms
/192.168.0.41:667 : 2ms
/192.168.0.43:1907 : 3ms
/192.168.0.41:666 : 2ms
/192.168.0.43:1908 : 2ms
/192.168.0.42:1908 : 1ms
/192.168.0.42:666 : 952ms
/192.168.0.42:667 : 952ms
/192.168.0.42:1907 : 952ms
/192.168.0.43:32121 : java.net.ConnectException: Connection refused
/192.168.0.41:32121 : java.net.ConnectException: Connection refused
/192.168.0.41:1907 : java.net.ConnectException: Connection refused
/192.168.0.41:1908 : java.net.ConnectException: Connection refused
/192.168.0.42:32121 : java.net.ConnectException: Connection refused

Cannot create the Datagram UDP socket.
It is likely that you do not have administrator privileges
on your filesystem. Re-run this tool using the 'root' administrator account.
______________

The thing is that I do not have any firewall and I`m only using Mac OS X with firewall disabled.
The cluster is connected internally with a GB switch and externally with another GB Switch. The client computer that I`m running DropSpot is directly on this "external" switch.

Regards
Lucas

Lucas, the problem seems to be with node 41. Can you try rebooting node 41?

If the problem happens again, can you copy the log file from /Library/MatrixStore/root/var/mxs/log/mxs.log on that node and send it to the info@object.... email address. Thanks!

Hi Lucas

Stop NFS sharing on node 41

Please note that switching off sharing using Server Admin does NOT turn off the service completely. You need to reboot the machine after doing that.

If that doesn't help please send output of a command:

netstat -a | grep LISTEN

launched on that machine.

Thanks

Hi Daniel,
you are omnipresent ehehehh!
I do have a NFS share on the node 41 (because this node is a Qmaster cluster controller, but it only consumes 2 processors). I`m aware that you do not support running any other service/server on Mxs nodes, but I`m really getting this running great despite that problem.
I stopped the Qmaster (and the sharing stopped). After that I killed NFSD service and tried (dropspot and Mxs Admin) and didn`t worked.
the output for the netstat is:

tcp4 0 0 *.1013 *.* LISTEN
tcp4 0 0 *.sunrpc *.* LISTEN
tcp4 0 0 *.terabase *.* LISTEN
tcp4 0 0 *.disclose *.* LISTEN
tcp4 0 0 *.ftp *.* LISTEN
tcp4 0 0 *.kerberos *.* LISTEN
tcp6 0 0 *.kerberos *.* LISTEN
tcp4 0 0 *.rtps-dd-mt *.* LISTEN
tcp4 0 0 *.49158 *.* LISTEN
tcp4 0 0 *.mdqs *.* LISTEN
tcp46 0 0 *.vnc-server *.* LISTEN
tcp4 0 0 localhost.pyrrho *.* LISTEN
tcp6 0 0 localhost.pyrrho *.* LISTEN
tcp6 0 0 localhost.pyrrho *.* LISTEN
tcp4 0 0 *.8826 *.* LISTEN
tcp4 0 0 *.8821 *.* LISTEN
tcp46 0 0 *.http *.* LISTEN
tcp4 0 0 *.8822 *.* LISTEN
tcp4 0 0 *.ssh *.* LISTEN
tcp6 0 0 *.ssh *.* LISTEN
tcp4 0 0 *.afpovertcp *.* LISTEN
tcp6 0 0 *.afpovert *.* LISTEN
tcp4 0 0 localhost.ipp *.* LISTEN
tcp6 0 0 localhost.ipp *.* LISTEN

Regards

Hi

As I wrote before

You need to switch off the NFS sharing using Server Admin.
After that you HAVE TO REBOOT machine

Killing the process is not enough because:
- it stops only the NFS daemon and not other services (i.e. portmapper)
- daemon will restart after reboot

D.

I`m sorry Daniel,
I don`t have Server Admin as my OSX it`s not Server version.
I could disable in terminal issuing "sudo nfsd disable" and reboot. I do believe I don`t have the full control over the NFS server for the fact that is Leopard and it doesn`t have NetInfo anymore. Probably Qmaster installation (along with compressor) it`s managing this NFS share in the background. I do stopped Qmaster and I do disabled NFS and reboot, but probably this isn`t enough.
I`m sorry that I couldn`t execute the commands that you passed me.
I`m still obsessed by the console messages that say:
rpc.statd[29] Failed to contact host macpro-8core3: RPC: Unknown host
rpc.statd[29] Failed to contact host macpro-8core4: RPC: Unknown host
These are the two other nodes. And I understand that RPC.statd and .lockd are services used by nfs right?
Regards

If you disable nfsd by:

sudo nfsd disable

it should work i.e. the service will be switched off permanently. Please reboot after that and send the mxs log file from that machine and he output of netstat -a | grep LISTEN again

Message

rpc.statd[29] Failed to contact host macpro-8core4: RPC: Unknown host

is not related to MatrixStore. It seems to be the problem with NFS setup.
To me it looks like server or client cannot resolve DNS names.
Try to use IPs instead when setting up NFS share.

I did a restart after that before replying to you. It doens`t changed, dropspot and mxs admin reports the same errors (error 200 and java.net.ConnectException: Connection refused).
I`ve rebooted one more time and the problem persists. The logs results I`ve sent through info@object-.... email.
Undoubtedly that rpc it`s not related to matrixstore, but as it`s reporting on console in intervals of 1min I could only thought that this could indicates an issue with the machine that could prevent matrixstore to work correctly, am I wrong?
I didn`t configure NFS share on this machine. The only thing I did was install Compressor with Qmaster (that uses NFS share). Unfortunately I can`t configure the NFS share, probably this will go away if I uninstall Qmaster, but that would messed up my system.
Regards

Hi Lucas

The problem, as you pointed out, is that Qmaster uses NFS sharing.
So even if you disable the NFS it still will be started by Qmaster service.

We don't support additional services on MatrixStore nodes because of such reasons.

So advice for now is:
Try to switch of QMaster on that machine temporarily, just to check if that's the source of the problems.
If yes, maybe wipe the MatrixStore cluster on all nodes completely and install it again but only on two machines which don't use Qmaster.

The MatrixStore will be fine with only two nodes, you just won't have additional protection and benefits of having third node (data regeneration).

Maybe at some point we will release a new server compatible with NFS sharing.

Regards

Hi Daniel,
Hummm that was my concern eheheh
I just forget to say that the two other nodes (42 and 43) are Qmaster nodes (service only) and doesn`t appear to have any problem. Maybe only being a Qmaster Controller (NFS server for the shared folder for the distributed process) it`s affected right? Or the logs indicates that the two other nodes are affected as well?
Surprisingly today (without restarting nothing) Dropspot gave me another error (and the bottom wheel indicates that the 3 nodes are connected -green). The error is: Exception on operation: Operation not allowed for the current user (error 301).
Mxs Admin still gives the same error of connection refused on the 3 nodes.
I`ve disabled "file sharing" and on Qmaster preferences i`ve started as service only and stopped (just to reset configs about being a controller) and restarted. The results are the same.
The strange thing are that in the mxs log it appears the RPC problem as well and apparently why mxs admin java.net.ConnectException: Connection refused.
the results are:

Nov 26, 2008 12:43:52 PM org.acplt.oncrpc.security.SecurityBackend omrpc_init_1
INFO: connection from + Socket[addr=/192.168.0.17,port=60948,localport=1907]/internal rpc client 1.0.0
Nov 26, 2008 12:43:52 PM org.acplt.oncrpc.security.SecurityBackend omrpc_init_1
INFO: connection from + Socket[addr=/192.168.0.17,port=60953,localport=1907]/internal rpc client 1.0.0
Nov 26, 2008 12:43:52 PM com.om.mxs.server.api.j invoke
SEVERE: Mac doesn't match for 2df4c61b-b683-11dd-bc32-8edc2c7b4e38
java.lang.SecurityException: Mac doesn't match for 2df4c61b-b683-11dd-bc32-8edc2c7b4e38
at com.om.mxs.server.security.SecurityManager$a.a(Unknown Source)
at com.om.mxs.server.api.j.checkAccess(Unknown Source)
at com.om.mxs.server.api.j.invoke(Unknown Source)
at $Proxy5.rpcSearch_1(Unknown Source)
at com.om.mxs.server.api.gen.mxsRpcServerStub.dispatchOncRpcCall(Unknown Source)
at org.acplt.oncrpc.security.SecurityBackend.dispatchOncRpcCall(Unknown Source)
at org.acplt.oncrpc.server.OncRpcTcpConnectionServerTransport._listen(Unknown Source)
at org.acplt.oncrpc.server.OncRpcTcpConnectionServerTransport.access$000(Unknown Source)
at org.acplt.oncrpc.server.OncRpcTcpConnectionServerTransport$1.run(Unknown Source)

I`ll after that, uninstall completely the Qmaster (and compressor on the node 41) from all nodes.
But I do have a feeling that all problems end up no the RPC problem that I`m having and I believe that there`s nothing to do with mxs. I`ve searched a lot about this error, but it seems that few peaple had this issue.
The goog news is that DropSpot is connecting with all the 3 nodes but if I try to search anything that error (above, different from initial one) appears.
The bad news are that Neither FTP, Dropspot works ok.
The error on FTP of the FCP Plugin is:

Connecting to ftp server...
Sending /tmp/fcs_db_backup-2008-11-26_at_0102PM.tar (504 MB) to ftp://localhost
Start at: Wed Nov 26 13:02:37 -0200 2008
Error =
ERROR: Couldn't send tar file to the FTP.
Script finished at Wed Nov 26 13:05:10 -0200 2008
Deleting the temporary tar file

And on the /Library/Logs/MatrixStoreConnect the error appear to be the same as dropspot:

INFO: Open connection - 127.0.0.1
Nov 26, 2008 1:02:37 PM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Login success - 2df4c61b-b683-11dd-bc32-8edc2c7b4e38
Nov 26, 2008 1:03:14 PM com.om.mxs.ftpserver.filesystem.ServerProxyImpl mkdir
SEVERE: Could not create directory object for fcs_backups due to Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:03:14 PM com.om.mxs.ftpserver.filesystem.MxsFileObject mkdir
SEVERE: could not make some of the dirs in /fcs_backups: Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:03:52 PM com.om.mxs.ftpserver.filesystem.ServerProxyImpl mkdir
SEVERE: Could not create directory object for fcs_backups due to Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:03:52 PM com.om.mxs.ftpserver.filesystem.MxsFileObject mkdir
SEVERE: could not make some of the dirs in /fcs_backups: Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:04:30 PM com.om.mxs.ftpserver.filesystem.ServerProxyImpl mkdir
SEVERE: Could not create directory object for fcs_backups due to Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:04:30 PM com.om.mxs.ftpserver.filesystem.MxsFileObject mkdir
SEVERE: could not make some of the dirs in /fcs_backups: Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:04:30 PM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Removing idle user null
Nov 26, 2008 1:04:30 PM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Close connection : 127.0.0.1 - <not>
Nov 26, 2008 1:05:08 PM com.om.mxs.ftpserver.filesystem.ServerProxyImpl mkdir
SEVERE: Could not create directory object for fcs_backups due to Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:05:08 PM com.om.mxs.ftpserver.filesystem.MxsFileObject mkdir
SEVERE: could not make some of the dirs in /fcs_backups: Exception on operation: Operation not allowed for the current user (error 301)
Nov 26, 2008 1:05:10 PM sun.reflect.NativeMethodAccessorImpl invoke0
INFO: Close connection : 127.0.0.1 - 2df4c61b-b683-11dd-bc32-8edc2c7b4e38

I`m sorry for importunating you all even that you doesn`t support other services, but IMHO this could help lot`s of people in the future as MXS are wonderful and FCP/FCSvr guys would be using it more and more.
Thanks a lot for all you help till now.
Regards</not>

I think only the node41 (controller) runs NFS service.
The other two are clients and use NFS share therefore they are not running portmapper (which is the source of the problems)
RPC errors appear because both NFS and MatrixStore are using RPC calls and both services want to launch portmapper.

Before uninstalling QMaster, could you launch netstat command on two other nodes and post the output?

Sure! Yes, only the node41 has NFS server share. The other two are clients, but what claims them as only clients are in the Qmaster pref pane, where I can setup a node to be service only or controller with services (the case of node 41). So in principle, if I chose services only on node 41 the NFS server would disappear right? The netstat of node 41 does show that nfsd it`s not running but only rpc.statd.
*Note that node 41 has qmaster stoped. Nodes 42 ans 43 are still running qmaster (as service - client - only). I will stop to see if anything changes.

Netstat from node 41:

tcp4 0 0 *.dawn *.* LISTEN
tcp4 0 0 *.intrastar *.* LISTEN
tcp4 0 0 *.terabase *.* LISTEN
tcp4 0 0 *.disclose *.* LISTEN
tcp4 0 0 *.ftp *.* LISTEN
tcp4 0 0 *.mdqs *.* LISTEN
tcp4 0 0 *.kerberos *.* LISTEN
tcp6 0 0 *.kerberos *.* LISTEN
tcp4 0 0 *.rtps-dd-mt *.* LISTEN
tcp4 0 0 *.49156 *.* LISTEN
tcp46 0 0 *.vnc-server *.* LISTEN
tcp4 0 0 localhost.pyrrho *.* LISTEN
tcp6 0 0 localhost.pyrrho *.* LISTEN
tcp6 0 0 localhost.pyrrho *.* LISTEN
tcp4 0 0 *.8826 *.* LISTEN
tcp4 0 0 *.8821 *.* LISTEN
tcp46 0 0 *.http *.* LISTEN
tcp4 0 0 *.8822 *.* LISTEN
tcp4 0 0 *.ssh *.* LISTEN
tcp6 0 0 *.ssh *.* LISTEN
tcp4 0 0 localhost.ipp *.* LISTEN
tcp6 0 0 localhost.ipp *.* LISTEN

___________

Netstat from node 42:

tcp4 0 0 *.dawn *.* LISTEN
tcp4 0 0 *.intrastar *.* LISTEN
tcp4 0 0 *.terabase *.* LISTEN
tcp4 0 0 *.disclose *.* LISTEN
tcp4 0 0 *.kerberos *.* LISTEN
tcp6 0 0 *.kerberos *.* LISTEN
tcp4 0 0 *.mdqs *.* LISTEN
tcp46 0 0 *.vnc-server *.* LISTEN
tcp4 0 0 *.rtps-dd-mt *.* LISTEN
tcp46 0 0 *.49157 *.* LISTEN
tcp4 0 0 *.49157 *.* LISTEN
tcp46 0 0 *.49156 *.* LISTEN
tcp46 0 0 *.49155 *.* LISTEN
tcp46 0 0 *.49154 *.* LISTEN
tcp4 0 0 *.49156 *.* LISTEN
tcp46 0 0 *.49153 *.* LISTEN
tcp4 0 0 *.49155 *.* LISTEN
tcp4 0 0 *.49154 *.* LISTEN
tcp46 0 0 *.49152 *.* LISTEN
tcp4 0 0 *.49153 *.* LISTEN
tcp4 0 0 *.49152 *.* LISTEN
tcp4 0 0 *.afpovertcp *.* LISTEN
tcp6 0 0 *.afpovert *.* LISTEN
tcp4 0 0 localhost.ipp *.* LISTEN
tcp6 0 0 localhost.ipp *.* LISTEN
___________

Netstat from node 43:

tcp4 0 0 *.dawn *.* LISTEN
tcp4 0 0 *.intrastar *.* LISTEN
tcp4 0 0 *.terabase *.* LISTEN
tcp4 0 0 *.disclose *.* LISTEN
tcp4 0 0 *.mdqs *.* LISTEN
tcp4 0 0 *.kerberos *.* LISTEN
tcp6 0 0 *.kerberos *.* LISTEN
tcp4 0 0 *.rtps-dd-mt *.* LISTEN
tcp46 0 0 *.49162 *.* LISTEN
tcp46 0 0 *.49161 *.* LISTEN
tcp4 0 0 *.49162 *.* LISTEN
tcp4 0 0 *.49161 *.* LISTEN
tcp46 0 0 *.49159 *.* LISTEN
tcp46 0 0 *.49160 *.* LISTEN
tcp4 0 0 *.49160 *.* LISTEN
tcp46 0 0 *.49157 *.* LISTEN
tcp4 0 0 *.49159 *.* LISTEN
tcp46 0 0 *.49158 *.* LISTEN
tcp4 0 0 *.49158 *.* LISTEN
tcp4 0 0 *.49157 *.* LISTEN
tcp46 0 0 *.vnc-server *.* LISTEN
tcp4 0 0 *.afpovertcp *.* LISTEN
tcp6 0 0 *.afpovert *.* LISTEN
tcp4 0 0 localhost.ipp *.* LISTEN
tcp6 0 0 localhost.ipp *.* LISTEN

Yes, by using only service and not controller on node41 MatrixStore should work fine.
The output of netstat looks better. What's the state of the cluster in AdminTool. How's Dropspot doing?

But unfortunately this is theoretically ehehhehehe
I`ve stopped the service on all 3 nodes. On node41 I`ve reset the Qmaster as service only and not controller. After that I`ve restarted the 3 nodes. All with Qmaster stopped and the same results of netstat posted above.
The same results for the DropSpot and Mxs Admin.
DropSpot gives me the error: Exception on operation: Operation not allowed for the current user (error 301)
All 3 nodes are connected (the bottom wheel is green).
On Mxs Admin I can connect to the cluster and Vault without any problems, on the status it`s all green (always was) and there`s no sign of malfunction unless I do "test connection" and the results are:
Output will appear here....

Trying to connect to supplied IPAdresses: [192.168.0.43, 192.168.0.41, 192.168.0.42]
/192.168.0.43:667 : 2ms
/192.168.0.41:1908 : 2ms
/192.168.0.41:666 : 2ms
/192.168.0.43:1908 : 3ms
/192.168.0.41:1907 : 2ms
/192.168.0.42:1908 : 2ms
/192.168.0.43:1907 : 3ms
/192.168.0.41:667 : 2ms
/192.168.0.42:1907 : 2ms
/192.168.0.42:666 : 2ms
/192.168.0.43:666 : 3ms
/192.168.0.42:667 : 982ms
/192.168.0.43:32121 : java.net.ConnectException: Connection refused
/192.168.0.41:32121 : java.net.ConnectException: Connection refused
/192.168.0.42:32121 : java.net.ConnectException: Connection refused

Cannot create the Datagram UDP socket.
It is likely that you do not have administrator privileges
_____________

Maybe it`s best to reinstall Mxs on 3 nodes and create a new vault with those services disabled?
Regards

But unfortunately this is theoretically ehehhehehe
I`ve stopped the service on all 3 nodes. On node41 I`ve reset the Qmaster as service only and not controller. After that I`ve restarted the 3 nodes. All with Qmaster stopped and the same results of netstat posted above.
The same results for the DropSpot and Mxs Admin.
DropSpot gives me the error: Exception on operation: Operation not allowed for the current user (error 301)
All 3 nodes are connected (the bottom wheel is green).
On Mxs Admin I can connect to the cluster and Vault without any problems, on the status it`s all green (always was) and there`s no sign of malfunction unless I do "test connection" and the results are:
Output will appear here....

Trying to connect to supplied IPAdresses: [192.168.0.43, 192.168.0.41, 192.168.0.42]
/192.168.0.43:667 : 2ms
/192.168.0.41:1908 : 2ms
/192.168.0.41:666 : 2ms
/192.168.0.43:1908 : 3ms
/192.168.0.41:1907 : 2ms
/192.168.0.42:1908 : 2ms
/192.168.0.43:1907 : 3ms
/192.168.0.41:667 : 2ms
/192.168.0.42:1907 : 2ms
/192.168.0.42:666 : 2ms
/192.168.0.43:666 : 3ms
/192.168.0.42:667 : 982ms
/192.168.0.43:32121 : java.net.ConnectException: Connection refused
/192.168.0.41:32121 : java.net.ConnectException: Connection refused
/192.168.0.42:32121 : java.net.ConnectException: Connection refused

Cannot create the Datagram UDP socket.
It is likely that you do not have administrator privileges
_____________

Maybe it`s best to reinstall Mxs on 3 nodes and create a new vault with those services disabled?
Regards

Don't worry about port 32121, it is used only for debug purposes, the output is fine. Netstat also looks good.

The problem with Dropspot is weird though. It might be the case that something went wrong during installation.

Could you please try to create a new vault using Admin and try to connect with Dropspot to it?
Please also send log files from all machines.

Reinstallation is the last step, better to know what happened :)

Unfortunatelly we won't be able to reply tomorrow, so if you want to install a new cluster - feel free, but please make sure you send us logs first

Regards
Daniel

Hi Daniel,
Thank you so much. But fortunately (or unfortunately) the problem doesn`t seems to be those Qmaster services. For my embarrassment after we got the error 301 on Dropspot (and on FCP plugin through ftp) the problem was that I didn`t create any user on Mxs Admin Vault. I was selection the Default user. Other thing that I didn`t understand is why the *.vault file for the new user that I`ve created has only node41 as ip addresses?
I thought that when I create a vault on Mxs Admin and whent to DropSpot and imported the *.vault file (even with the Default user) things would work ok. I couldn`t delete vaults (it`s grey on mxs admin), but I could add a new one, but it didn`t work, only after I`ve added a new user to this new vault and selected the import new vault on DropSpot with the *.vault file for the new user that everything worked (even FCP bkp Plugin)!
It`s a bit confusing I`m sorry about that.
I`ve restarted Qmaster (as a controller) and apparently things are working ok. I should do more testing, do you recommend any?
Thanks a lot again and sorry for the confusion and for my clumsy way....
Regards

Hi Daniel,
just to make it stick to this post:
It is nfsd service that causes the error 200 on dropspot (and on FCP plugin). You were right from the beginning! Just some mysterious things that got stuck even after lot`s of reboots. So in principle it`s not possible to use a MxS node as a Qmaster Controller (even quickcluster) because it uses nfsd service for the share volume between qmaster nodes.
The error 301 was very strange because it appears that the *.vault file created with the new vault on Mxs Admin doesn`t works. I could only get things working by creating a new user on any vault and using the new *.vault file to import in dropspot or by entering the values manually.
The strange thing I`m still concerned is about the IP Addresses in the *.vault files created by Mxs Admin. They always have one IP (one node) and not the range (in my case 3) even then the cluster is with the full range in Mxs Admin and all nodes are ok. I`ve changed manually the IP Address of the vault on DropSpot. Or am I doing something wrong?

Any suggestion about still trying to use Qmaster controller together with Mxs?
Thank you very much, you guys are wonderful for support (and building disk-based archived solution)!
Regards

Yes, I'm glad we reached at the same conclusion ;)
At the moment it's not possible to use both QMaster Controller and MatrixStore on a node. As I wrote few posts above - maybe one day we
will support that but it's rather unlikely because we would not have control over machine anymore, we wouldn't know how processes interact, what is the CPU load during encoding etc.
So basically it can be a shot in a foot

My advice:
If you have a spare machine usu it as a QMaster controller and the rest of the nodes as clients. MatrixStore should work fine on them I guess.
If you don't have a spare machine uninstall MatrixStore from all of nodes and install it only on two - two node cluster.

About error 301.
That's reallly strange as Dropspot and plugin should work fine with default user. I don't know why cluster complains about it. Maybe cluster was not installed properly (QMaster) and complains because of security problems on default users. Did you send log files on failled and succesful connection to the vault?

About IPs in the vault file:
That's a bug in AdminTool which has been fixed some time ago. I believe if you download the latest version of MatrixStore software it should be fine.
In the previous versions you have to add IPs manually - as you did.

Thanks for nice words

Cheers

Hi Daniel,
Yes, I have to agree with you that Qmaster (as a controller) it`ll be not stable in a MatrixStore cluster.
I`ll transfer the Qmaster controller to a spare mac (probably a mac mini or a old powermac g5) and leave the 3 MacPro 8 cores as MatrixStore with Qmaster services only.

I`ll try to uninstall completely the MatrixStore on 3 nodes and reinstall it without the Qmaster running as a controller. We`ll see if during the installation something got messed. Yes I did send the log files on failed and on successful (with a new user) I didn`t, do you want me to send to info@...?

That`s very strange, because I`ve completely uninstalled Mxs on 3 nodes and installed a fresh one with the latest release (2.1.18). Maybe some files wasn`t deleted with the uninstall script?
Regards