Default permissions differences on SFTP mounts (inheritance/umask?)

Hi Jeff

Good idea, I just tried it out with cyberduck and it behaves the same as the local ssh'ed connection does.

Expandrive is still not setting the same default permissions as either the local ssh'ed in test or cyberduck.

[gerk@gemini ~]$ ls -lah test*
-rw-rw-r-- 1 gerk gerk 0 Jan 6 16:17 testcyberduck
-rw-rw-r-- 1 gerk gerk 0 Jan 5 18:19 testlocal
-rw-r--r-- 1 gerk gerk 0 Jan 5 18:19 testremote

I just tested on another server (with a different umask setting). All 3 worked as expected so it's definitely something that's specific to how my server is setup ... but it's still odd that cyberduck and expandrive are somehow setting different default permissions ... is it possible that expandrive/MacFUSE is not actually reading what it's supposed to set and just setting a "standard" default setting (0022) instead of what the server is set to use (0002)?

llvjcom@llvj [~]# ls -lah test*
-rw-r--r-- 1 llvjcom llvjcom 0 Jan 6 15:21 testcyberduck
-rw-r--r-- 1 llvjcom llvjcom 0 Jan 6 15:23 testexpandrive
-rw-r--r-- 1 llvjcom llvjcom 0 Jan 6 15:21 testlocal

llvjcom@llvj [~]# umask
0022
( default should be -rw-r--r-- , and it is )

gerk@gemini ~]$ ls -lah test*
-rw-rw-r-- 1 gerk gerk 0 Jan 6 16:17 testcyberduck
-rw-rw-r-- 1 gerk gerk 0 Jan 5 18:19 testlocal
-rw-r--r-- 1 gerk gerk 0 Jan 5 18:19 testremote

[gerk@gemini ~]$ umask
0002
( default should be -rw-rw-r , and it is not )

It's a head scratcher!

Ok .. one further regression. It worked ok in 1.3.2 (at least on my powerbook which I hadn't updated yet), but it failed as above with 1.3.3 on the same machine after updating:

[gerk@gemini ~]$ ls -lah test*
-rw-rw-r-- 1 gerk gerk 0 Jan 6 16:17 testcyberduck
-rw-rw-r-- 1 gerk gerk 0 Jan 6 16:40 testexpandrivePB-1.3.2
-rw-r--r-- 1 gerk gerk 0 Jan 6 16:46 testexpandrivePB-1.3.3
-rw-r--r-- 1 gerk gerk 0 Jan 6 16:43 testexpandriveWS-1.3.3
-rw-rw-r-- 1 gerk gerk 0 Jan 5 18:19 testlocal
-rw-r--r-- 1 gerk gerk 0 Jan 5 18:19 testremote

[gerk@gemini ~]$ umask
0002
( default should be -rw-rw-r , and it is not )

Ok last one for a while (I promise) :D

Looks like a MacFUSE bug for sure, I just tried with a mount made with sshfs.app (which also uses MacFUSE) and got the same result as Expandrive:

[gerk@gemini ~]$ ls -lah testsshfs
-rw-r--r-- 1 gerk gerk 0 Jan 6 17:09 testsshfs

Also another user has reported the same issue on the MacFUSE google group (with no responses yet).

http://groups.google.com/group/macfus...

Hello Mr. Gerk,

I think Expandrive/Macfuse is launched by launchd and that it therefore inherits the umask setting of launchd. But then the server-setting is also important. I think the masks are "ORed" i.e. nor server nor client can lax the permissions for the other.

For example when I started using Expandrive I was getting "-rw-r--r--" for files created on the file-server. I wanted group-write so I configured launchd to umask 002 with "echo 'umask 002' | sudo tee /etc/launchd.conf". In the original configuration for leopard there is no "launchd.conf"-file and the umask is then defaulted to 022. This worked fine and files created through Expandrive became "-rw-rw-r--". (Noto bene a restart is needed for changes to launchd.conf to take effect!)

Now today I wanted to make the default permissions more strict, removing all permissions for "other", i.e. umask 007. My first instinct was to try putting a umask command in the .profile-file on the file-server (like the guy at MacFUSE google group) however I learnt that none of the standard login-profiles are executed during sftp-login. Instead I found that on my file-server (the luvely bubba|two) the sftp-server is started with a wrapper script at /usr/lib/openssh/sftp-server.sh and in there was a umask 002, changing that to umask 007 I got the preferred result. I.e. new files created through Expandrive is now "-rw-rw----".

Hi daEelz

Interesting ... but that doesn't explain why Expandrive 1.3.2 behaved correctly and 1.3.3 doesn't and why other SFTP clients behave correctly.

Also I just looked on my server and there is no sh script that launches my sftp-server, it's handled in sshd_config -- Subsystem sftp /usr/libexec/openssh/sftp-server -- which is a binary and not an sh script.

Interesting about changing the permission of the local launchd -- but that is not a good solution! I don't want to have to change the default permissions of all of my locally launched daemons to include groups on my workstation in order to get it on my server :/ I'm pretty sure at this point it's a bug/behavioral change with MacFUSE 2.0 as 1.7 seemed to handle it fine.

Just to note for regression purposes, the problem still exists with Expandrive 1.3.4b2

-rw-r--r-- 1 gerk gerk 0 Jan 9 18:07 testexpandriveWS-1.3.4b2

I regressed with today's release, 1.3.4 .. still has this issue.

-rw-r--r-- 1 mark 500 0B 13 Jan 17:24 testexpandriveWS-1.3.4

bump ... any news on this one Jeff? Haven't heard anything back from you since you asked about trying it in another sftp client 9 days ago ...

Bump ... ok this one is getting pretty frustrating guys, no responses at all from the Expandrive folks in 17 days.

This problem is causing me a ton of chaos here, having to manually fix every single file that Expandrive touches (and breaks). Guess it's time to move on to another app if this can't get resolved, sigh.

Yes I also can pretty much confirm this is a MacFUSE issue as I stated earlier in this thread. I tested with sshfs.app and had the same results as expandrive. MacFUSE 1.7 worked as expected (reading and setting the umask properly), MacFUSE 2.0 did not work properly. This happened with both Expandrive and sshfs.app (both of which use MacFUSE).

I was hoping that you guys, since you use MacFUSE for your core could cofirm this and take it up with the MacFUSE folks ...

The problem is that us end-users can't see the actual call being made, so it's not much use for us to file bug reports with the MacFUSE folks ... I know that at command line at least there's a way to pass the umask, but Expandrive seems to use other 3rd party python stuff, so who knows what's going on under the hood :)

you said: "My apologies for the delay in responding. I must admit, I'm not sure why there should be any difference between version 1.3.2 of ExpanDrive and later versions"

1.3.3 introduced MacFUSE 2.0 usage.

"I traced the calls passed from MacFUSE into the ExpanDrive filesystem driver, and it is in fact giving ExpanDrive the "correct" permission bits, relative to the local filesystem's umask. However, with my test setup (OpenSSH_5.1p1) these are getting clobbered by the SFTP server during file and directory creation, presumably because it is applying its own (remote) umask to the mode word we send."

This is the exact behaviour I'm LOOKING for! You have it opposite to what I've reported here. All of the other SFTP clients honor the remote umask when you make a new file -- which is correct behaviour. Expandrive is using the local umask and ignoring the umask set on the server.

*sigh*

You might want to re-read all of the above posts. If I understand what you are saying then it means that Expandrive is currently working correctly for you but not for me. Please test with the examples I've given in this thread -- try with a remote umask of 0002 (on your server). This is a very common setup (to give group access to new files by default) and the exact issue I'm having and can reproduce the behaviour on multiple servers/different OSes.

I feel strongly that it should honor the remote umask, as it did up until 1.3.3 was released ... that's the whole point of having a umask set on the remote server, so that the files created on that server have the default permissions that the server assigns. Honoring the local umask is not helpful as it's not a local filesystem (even though it's mounted locally).

To me this is a bug, as again, it worked as expected (at least as I expected) until 1.3.3 came out when there was a behavioral change. All of the other popular SFTP clients honor the remote server's umask.

If it doesn't honor the remote umask it's going to cause grief for a lot of users (like me) who need things set with specific permissions on a per-server basis. Having to adjust your local umask in order to accomplish that is not viable (what happens when you need to connect to 2 different servers at one time that need different default permissions?)

Also when you do need things like a umask of 0002 on a remote server you shouldn't have to compromise your local umask by enforcing full group access to all of your files on your local workstation to accomplish this!

Our intended behaviour for ExpanDrive was to honor the local umask, not the remote one, since it is supposed to present a remote filesystem as if it were connected locally. The test cases you showed indicate that it's not doing that, at present, which---as I see it---is something we need to work around. I would certainly agree that we should honor either the local umask or the remote one! :)

My tests show that it is in fact honoring my local umask and not the remote one, I think you still have things backwards here. If you folks are taking the opinion of treating the mounted filesystem as a if it were a local filesystem (which it most obviously is not) then it's the wrong approach without a doubt. it is a remote filesystem and the files and folders created on it must be compatible with the way the remote server is setup, first and foremost. Being able to mount it locally is nothing more than a convenience and shouldn't be treated as the primary usage for the filesystem.

At this point I think I'm just going to toss my hands into the air and walk away and go find myself another SFTP client. Sorry if it sounds abrupt, but Expandrive's first and foremost function is as an SFTP client ... and as a client it should be honoring the permissions as set by the server, not setting it's own policies and breaking everything it touches on the server.

This issue shows up when using sftpdrive from Windows to Linux as well. We were thinking of purchasing 20+ licenses but when I edit a file it reverts the file permissions to the umask. I agree that it should honour the umask, but it should not change the umask of an existing file. This is very painful if you edit scripts from Window and it takes off the execute permissions.

Unfortunately this behaviour makes it unusable for me. I really like this product and I'd like to see it become our company standard.

thanks

Hi Gordon

Yes I see this exact same behavior as well, any file I edit the permissions get stomped which also makes it unusable for me.

In response to Jeff, that would work for some situations but it is very restrictive. It does not appear that there is a temp file created on the sftp share. My issue is this: If I want to edit a single file then I ssh to the server and use vi. If I need to do mass changes and copies and comparisons between the files on different servers, I cannot easily do that in Linux, so I use Windows tools like BeyondCompare and UltraEdit. If I copy a few hundred files, I can't assume that the umask is going to be the same for all of them. I'm sure it would be a tedious thing to impliment in your code, but I think there needs to me a permisions option that reads each file and writes it with the destinations umask. (maybe with a warning that says this option might me slower :-)

M: Any news yet?
We are also waiting for this to be fixed (actually we degraded to an older FUSE version (without using ExpanDrive). Maybe it is an idea to always respect the remote umask (default), unless a user specifies a different umask for that drive in the expandrive drive manager.

I'd like to add also that in the case of an existing file the "new" file (i.e. the file that is there after you edit and save) should have the same permissions as the previous version did.

Have the ExpanDrive folks given up on this one?

Yes, this is really frustrating to see that nothing happend to resolve the umask issues!

ETA? this problem is preventing from purchasing your software on a wide scale

Jeff,

Thx for the new beta, let me explain a little bit more on what/how we tested.

I just wrote a script to generate 64 test files with different permissions (owner/group/all with all different permissions 4,5,6,7).
My conclusion is that this still does not work as expected.
I tested this with Ubuntu 8.0.4 Server, and OpenSolaris 2008.11 both running OpenSSL 5.1.

Example 1
Transferring a file from local to the SFTP server, with a server umask of 017:
local -rwxrwxrwx (777)
server -rwxrw---- (760)
This works as expected

Transferring a file from local to the SFTP server, with a server umask of 007:
local -rwxrwxrwx (777)
server -rwxrw---- (760)
Here we would expect the server file to have a it's permissions set to 770, instead of 760.

Example 2
Transferring a file from local to the SFTP server, with a server umask of 017
local -rw-r--r-- (644)
server -rw-r----- (640)
As you see the 'all/other' bit is correctly set to 0.
However, the server does not automatically apply the 'write bit' to the group.
I would expect it's permissions to be set to (660), since the umask is set to 017.
But this is not the case, the server seems to respect the fact that the local file did not have and group write permissions.

Why do we need this?
The reason for us to need such a feature is as follows (example):
We have a shared server, with two users, both users belong to there own group, but also to a shared group named projects.

drwxrws--- 7 projects projects 4.0K 2009-02-24 15:41 ./
drwxrw---- 2 gawin projects 68B Feb 24 16:45 files/
-rw-rw---- 1 gawin projects 0B Feb 24 16:45 readme.txt
-rw-rw---- 1 gawin projects 0B Feb 24 16:45 install.txt
-rw-rw---- 1 chris projects 0B Feb 24 16:45 todo.txt

In this situation chris can CRUD all of gawin his files. And if chris uploads something, then gawin can also CRUD chris is files.
However, if user john (who does not belong to group 'projects') wants to CRUD something, then this would be not posible (which is exactly what I expect/want).
As you see in example 1 and 2, this however does not work using sftp. Since most of gawin his local files have a umask of 0022 (644). And therefor end up at the server side have 640 permissions.
Which causes chris not to be able to change/update or delete the file, only to read it:

-rw-r----- 1 gawin projects 0B Feb 24 16:45 uploaded.txt

Advanced situation 1 (SGID bit)
We have some more advanced situations where the folders have different masks on the server (public www folder, www-upload etc).
So this is where it becomes more difficult, in this case the sftp client should respect the folders permissions when uploading a file.
We achieve this using the SGID bit, which works great when using SSH. However, SFTP does not seem to respect this.

Advanced situation 2 (Sticky bit)
Then there is also the situation where we use the sticky bit, Which causes the same problems as the SGID bit.

Having the same issue as mentioned through out this thread. I am on version 1.7.9. Any updates? I know webdrive negotiates serverside permissions correctly. However Filezilla produces the same results as SFTPDrive.....

I apologize for my lack of knowledge but I may be restating what has already been covered. If I upload a file via sftpdrive the resulting permissions on the serverside is rw------- only.....if I right clck on the drive > Change Creation Mask and check all check boxes and upload i then get a rwx-----

Earlier today I had a discussion with some OpenSSH guys.
There reaction was something like: Don't expect SFTP to correctly handle permissions, since it always gives problems.
If you want to change that, feel free to write a patch for OpenSSH. (great!)

If I look into the specifications (assuming that these are the specs that are used?):
http://tools.ietf.org/html/draft-ietf-secsh-filexfer-13#section-7.6

In specification says that the file atributes 'ATTRS' can be defined. Which can be used both ways, server to client, and client to server.
On of the attributes are the file permissions.

(second value is the number in octal):


S_IRUSR 0000400 user-read
S_IWUSR 0000200 user-write
S_IXUSR 0000100 user-execute
S_IRGRP 0000040 group-read
S_IWGRP 0000020 group-write
S_IXGRP 0000010 group-execute
S_IROTH 0000004 other-read
S_IWOTH 0000002 other-write
S_IXOTH 0000001 other-execute
S_ISUID 0004000 suid (set uid)
S_ISGID 0002000 sgid (set gid)
S_ISVTX 0001000 svxt (save text)

The specification states: "The server SHOULD NOT apply a 'umask' to the mode bits; but should set the mode bits as specified by the client."
Which basically would mean that setting a umask on the server side would be useless.

The specification also states: "The client MUST apply an appropriate 'umask' to the mode bits before sending them."
So, if I read it correctly, ExpanDrive should apply this appropriate umask before sending the permissions attribute.

Although the permission bit is better explained over time, it was included since day one. So the question that remains: Is OpenSSH or ExpanDrive not handeling the permission bit correctly?

Has this issue been resolved with the 2.x release?

No (at least not in version 2.0.2)

Well, I was thinking about buying Expandrive, but then I ran into this exact same problem. In recognition that it's taken the company 7 months to get absolutely nowhere with it, I guess that I won't buy the product.

BTW, you could just solve this by adding a field to the "Configure drive" screen where the user can optionally enter a mask to be applied by Expandrive after every write to the remote filesystem.

Are there any plans to add that functionality (setting the desired umask per configured drive) to the Mac version of ExpanDrive in the near future? It is most annoying and counterproductive, that ExpanDrive does not respect the server-side umask nor enable one to set it within the client itself. It forces one to constantly chmod the files and creates countless issues when collaborating with others on a joint server.

Just a note here for anyone else who didn't figure this out and was searching for answers:

I'm using Expandrive 1.8.3 on windows, and you can set the umask by right clicking on the drive name in the config window and choosing "Change creation mask".

This allows you to set the permissions for all uploaded files, and works perfectly on windows -> mac, and windows -> unix servers.

If you need different permissions for different files map the drive a few times and copy files across to different drives for different stuff (eg. copy your cgi-bin directory across on the +x mapped version of the drive.)

Just upgraded my license to the new 1.8.3. Still can't get group permissions when uploading the file. I thought maybe with the new version it would work. By default no options are checked. So when uploading a file with the default settings the file shows rw permissions at the user level ( -rw------). No matter what combination of options I select the most permissions I can get is exec at the user level (-rwx-----). I love the stipped down aspect of SFTPDrive but unfortunately we need our files to be written with rw permissions at the group level. We had to go with WebDrive.

FWIW you can still only set the default umask on the windows client, the OSX one doesn't have this option.

Yeah...no success though for me...the UMASK UI is there but doesn't seem to work.

+1 vote to honor the server umask by default. This is just common sense.

I'm with other posters who have suggested that how ExpanDrive works right now isn't good enough and that there should be the option to explicitly set permissions on newly created files on a per host basis.

This is a very common usage scenario for SFTP and is supported by virtually all SFTP clients (including Cyberduck).

I was able to fix this issue indirectly by using rssh, which allows you to set the umask for the SFTP server. It's confirmed to work with 2.x of ExpanDrive on Mac and 1.x on Windows.

I'm not particularly happy with this solution, because from what I saw reading various newsgroups and forums, the onus for setting permissions and defining the umask rests with the client. I shouldn't have to do this server side.

P.S. For the record, we do have a site license for ExpanDrive, so I am trying to make it work for us...