Loading Profile...
Remote usability testing on anyone with a Mac + iSight
Now that you've managed to do away with the need for external cameras, it seems possible to test users anywhere so long as they have a Mac with an iSight camera. A "microapp" that is sent to the user to be tested that does the recording and sends the data back to the home app, perhaps?
I’m really excited!
22
people like this idea
I like this idea!
Tell me when this idea gets some attention.
The more people who like this idea, the more it gets noticed.
The more people who like this idea, the more it gets noticed.
The company is considering this idea.
The best point from the company
-
That's a great idea and one we've thought about. However it's quite complicated to do and would require a good deal of bandwidth to stream the results. It could also complicate the interface so we'd probably reserve it for a pro version of the app if we ever do one.
The company and 1 other person think
this is one of the best points
-
Inappropriate?That's a great idea and one we've thought about. However it's quite complicated to do and would require a good deal of bandwidth to stream the results. It could also complicate the interface so we'd probably reserve it for a pro version of the app if we ever do one.
The company and 1 other person think
this is one of the best points
-
Inappropriate?Hey Andy, thanks for responding so quickly!
My thought wasn't to necessarily stream the results as they happened, but rather to record them on the user's local machine and send back the final video post-test. It'll still take a good amount of bandwidth to send the video and/or data back, but it could be spread out over a longer period of time, I suppose.
I, too, imagine this feature would be pretty complex to implement, but I'm a little concerned about it potentially only being included in a "pro" version of the app. I know a number of people - myself included - who either work for small start-ups or develop small apps on their own, and the ability to do remote user testing would be an INCREDIBLE help. While it would be inefficient to bring 5 people in to test a web app written by one guy in 10 hours, it would make sense to email 5 people and have them spend 15 minutes on their own machines testing. Likewise, while $50 might be an acceptable price point to the lone developer, anything more might not be possible.
Forgive me for being verbose! I'm sure you all have covered these issues already. Personally, I love the way the app looks and operates right now but I don't have a use for it, since anything I create is going to be so small as to not warrant sitting someone down in front of a machine to test it. But, if remote testing were possible, I'd buy it in an instant. This seems to be a weird feature that would add complexity and cost, but would give the most benefit to the developer with the least to spend.
Either way, good luck with Silverback. It's a fantastic app and I'm sure it will be wildly successful, with or without the remote ability. ;)
I’m confident
1 person thinks
this is one of the best points
1 Comment Add a comment
This is one of the best points
-
You'll be surprised how cheap and easy it is to sit people down in front of your apps to test. You can find people thought ads on notice boards, requests on mailing lists or through Twitter. You can pay people in gift vouchers, movie tickets or wine and chocolate, so it doesn't have to cost a fortune. In fact I generally find it easier (and more accurate) to be in the room with people than to manage a remote test. -
Inappropriate?Streaming would be extremely helpful. Great idea and worth the extra money, if that is an upgrade.
-
Inappropriate?Even if I just had the audio streaming and ability to stop and start remotely, that would be a good place to start, and allow me to review the quicktime after the session. I need to use Silverback for applications testing, more than web. Filetransfer of the results, rather than stream them is suitable.
-
Inappropriate?Andy - Awesome app. I've been looking for a replacement for Morae on the Mac forever. Just wanted to give a +1 on the remote management idea. I'm not so concerned about having this available via the web, but I just like to be out of the room when the test is happening so it can happen on my local LAN. I'd just like to be able to "observe" what's going on from my machine and record chapter markers and so on.
1 person thinks
this is one of the best points
-
Inappropriate?To do a proper usability test you really need a moderator to set the tasks, prompt the subject for their responses, and help them out if they go wildly off track. As such, there is no substitute for running a test in person.
You can run remote tests, but you still need a moderator involved. To bake this into the app you would therefor need to stream the audio, screen activity and preferably iSight video back to the moderator and allow them to communicate with the subject.
You could attempt to do this currently by using Silverback with some form of net meeting software. However we've not tried this yet so are not sure how it would work.
I agree that remote management would be a good thing to have. However it would be quite a significant addition from both a UI and engineering perspective. You'd need to create a client/server model with different components and this could fundamentally change Silverback from the nice little guerilla testing app into something more akin to Morae.
It's definitely on our wishlist, but we'd need to consider this carefully before implementing it as it would significantly increase the cost and complexity of the app.6 Comments Add a comment
This is one of the best points
-
Yes and no. I use iChat on the remote users screen to issue instructions on the next task and the user records a screencast And sends it to me of each task. Cumbersome. With silverback I could pause for the instructions and record the user in addition to the screen
-
That sounds like an interesting approach. However I'd still want to see what the user was doing on screen as I'd find it very confusing otherwise.
-
Arguably, if the user has gone wildly off track, then that indicates a problem in the product that's being tested (ok, gross generalization.) As long as the subject lets me know that he's completed the instructions, and I can pause and give new ones or pause and ask for feedback, I'm ok - I don't need streaming, but I do need to be able to test with far-flung users.
-
Arguably, if the user has gone wildly off track, then that indicates a problem in the product that's being tested (ok, gross generalization.) As long as the subject lets me know that he's completed the instructions, and I can pause and give new ones or pause and ask for feedback, I'm ok - I don't need streaming, but I do need to be able to test with far-flung users.
-
I think my point was more that it's very difficult to tell what a user is doing just from what they say that their doing. You really need to see what they are doing. Otherwise you may end up having to interfere with the test to often just to ensure you're on the same page so to speak. I'm not saying it's impossible, just more complicated and open to more interpretation and potential problems.
-
Agreed Andy; what people say and what they actually do are often completely different (and that concept extends way beyond usability testing...its human nature).
-
Inappropriate?I agree that streaming would be a great feature for a Pro Version. As it is, this application could work for +50% of our needs. For those other 50% we need to have the ability to have remote viewing. For us, the most important thing is to have a viewing room (often next door) where other team members and often the client can listen to the interviewer and and watch the test subject.
-
Inappropriate?While we wait for a Mac-friendly remote testing tool, I can recommend UserVue for remotely moderating and recording test sessions in Windows. It works well, though it only records screens, no external video. -
Inappropriate?A simple solution is to use iChat, and assuming the participant is using leopard, have them share their screen with you. As long as you start recording the session before the screen share happens, it should work perfectly.
I tried this out the other day with surprisingly good results.2 Comments Add a comment
This is one of the best points
-
Interesting idea. I've not tried this before so it may be worth giving it a go.
-
I just tried using iChat screen sharing for remote viewing. I decided that using Bonjor was the best idea so the data would remain on our high-speed LAN and so that there were no firewall issues. For some reason the screen sharing was crashing after about 2 seconds, every time. I'm not sure why- maybe a conflict with Silverback and iChat wanting control over the camera? Any we ended up using GoToMeeting (something we use for web meetings) and a conference call line. Not elegant but effective none the less.
-
Inappropriate?If both parties are running Leopard, it my be worth it to look into the "share remote desktop" feature in iChat. I have used it with my sister, and I can not only see everything she's doing on the desktop, but she can hear what I'm saying while she's trying to do something. Since they have already found a way to do the desktop sharing, perhaps the video from the isight could be recorded and transmitted separately, so you could see the desktop in real time, just not the video.
EMPLOYEE
