Help get this topic noticed by sharing it on Twitter, Facebook, or email.
I’m Hopeful
Cullen Brimhall (Help and Support Product Manager) November 08, 2010 15:48

Update from Product Management - Pilot vs Beta

Posted on behalf of Robert Kehrer:

I'm the product manager for beta search. While the whole team of engineers, UX designers, etc all participate in reading feedback and defining priorities around improving the product - ultimately the responsibility resides with me for the entire search experience (I don't manage other parts of the site like the blog and wiki).

I'm searching for ways to say this next part more broadly and clearly to better address the "We're not being heard" comments, but since this is an active thread, here's a few thoughts that I hope are of value...

1) The users on this site ARE being heard.
We all read the feedback. Examples of searches with detail about what was searched for and how the results were unexpected are captured and used to tune the search engine. Feature requests that clearly outline how a user desires to get a task done on the site are captured in our future feature plans, designed around, tested with users. Clear constructive criticism with examples and suggestions have been extremely helpful in prioritizing and defining the site. Users who contribute little more than negative adjectives and make it a mission to stir up angst are not. For those who are not comfortable trying out an unfinished work in progress we are leaving up Pilot and www.familysearch.org for their daily work.

2) Pilot cannot handle anticipated collection growth
Pilot served user's needs well. However, the technology world and the scope of FamilySearch' plans to deliver records has changed radically over the last 6-8 years since work began on Pilot. Most of the technology under the user interface is not capable of moving forward with FamilySearch. I've outlined the reasons at the top of the post referenced below.

http://getsatisfaction.com/familysear...

3) Pilot is NOT going to be turned off until Beta is ready
We've invited users to come check out the new house we're building, it's not ready but some things are worth a look and we'd like your opinion as we build. We've let you know that from a technology standpoint Pilot is irreparably broken, will not receive any more investment and will someday go away. However, we're not going to remove the house you live in (Pilot, www.familysearch.org) until the new house is complete. Some of Pilot's limitations are exposed (like collections that can no longer be displayed & no new collections) We're trying hard to keep these minimal on Pilot and evolve beta into a replacement as quickly as possible.

With that said the words "ready" and "complete" are up to interpretation. Here's some perspective. We've identified a set of features (filters, control over search parameters, improved UI, improved search engine tuning, etc) that must be delivered before shutting Pilot down. Additionally we will be watching the movement of users from Pilot to Beta. When the remaining daily unique users on Pilot are sufficiently small, and the key features have been delivered, we'll probably ask "What compels you to keep using Pilot?". Based on the answers we may choose to address a few remaining needs before turning Pilot off. No date has been set to turn off Pilot & any conflicting statements to this are erroneous.

4) Development strategy & Development Time
Traditional development plans call for a long period of development, a short beta test just prior to launch, some quick fixes and a release. Unfortunately this allows for a relatively short timeframe to incorporate user feedback (every product I've ever seen has gotten stuff wrong at ship). Traditional development timelines are not required when a product isn't boxed and subject to traditional release costs and new models are being explored.

Unlike some software projects that are updated once every year or so, our development cycles are fast so that we can adjust & respond rapidly to user feedback. We release every month with a few new features or enhancements. Some features that seem easy to a user (especially when they are already out on Pilot) took many months of work on Pilot and take us time as well. We are using as much of Pilot as we can (don't want to reinvent wheels), but the most valuable and portable part of Pilot (the learnings) will be represented.

Sometimes we get a feature wrong (ex. we're working on an improved search form layout) - our rapid development process enables us to fix it in months rather than years. More often we have either not really implemented a feature yet (like filters & collection specific search) or we have implemented it functional but basic so we have something to iterate on (like Catalog search). It's totally understandable that if a user is standing in a kitchen that is nothing but studs and ductwork they are not going to be very productive even though there may be a small plastic sink in the corner. The plans call for granite counter tops and gold fixtures.

Part of what we learned through this experiment of building a site in public view is the importance of putting in place the tools and organizational processes to continually message around roadmaps and feature designs. This is still an evolving feature of the site - but I think we're getting better.

5) Who is the team
We are not professional nor accredited genealogists, but we are technologists who have a pretty deep understanding of and passion for genealogy. Many of us are under mandate to do our own genealogy as part of our jobs. We learn from and test designs with the many many experts surrounding us at FamilySearch. We also study and test our future features and designs with users where they gather at conferences and Family History Centers.

I am very confident that we are on track to build something that is superior to Pilot, even if at doesn't feel that way right now when there's no drywall or fixtures in some rooms. We'll get there.

-Robert
Reply
  • If pilot is going to be retained until Beta is completed, then take off the warning messages at pilot that it will be removed soon. Please stop the data transfers from pilot to beta. Too many images are gone from pilot and trying to retrieve them from beta is a pain in the butt.

    Please post your current message as a reply to each of the problem areas noted on line (see link A) and send a memo to every employee, tester and volunteer. The angst generated recently has been unbelievable. sending out your message to everyone would allow every one to take a step back and relax for a bit.

    Thank you.

    A)
    http://getsatisfaction.com/familysear...
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • Is there some reason why the beta site could not look and act more like the pilot site? The larger font and additional spacing means more scrolling. I realize there are people with vision problems, but there are ways to individually increase font size on your computer. Some web sites have a selection of font sizes on the screen.

    Also, I found, by accident, that you have to have done a search and then click on feedback to get a field to enter an opinion. If you are at the home page, and haven't done a search, and click on feedback, you see what others have said. The only field to enter something is "Ask A Question" which did not seem like the appropriate place to express an opinion.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m excited
    Thank you for explaining where you are at and where you hope to be in the future.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • Cullen and Robert,

    Thanks for this update. I agree with Tim that the update should be posted on each of Topics (Threads) where a bulk of the angst is expressed. Hopefully, most people wanting to express concern will then see this update and have there worries lessened.

    I look forward to seeing the road map to future designs and features.

    I would suggest modifying the update by changing "Pilot is Broken" to say "Pilot Cannot Handle Anticipated Collection Growth" or something like that. Pilot is still the best genealogical search tool available. More users would not consider it broken, so the phrase sounds over-reaching.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • The link under "Pilot is Broken" doesn't work.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m grateful
    Many thanks to Cullen and Robert for a helpful explanation. We appreciate all you're doing!
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m happy
    I too am very grateful for all that Family Search is doing for genealogy.

    Could I ask three things if they haven't been asked already with regard to the Irish Civil Registration Indexes on Beta please..............

    1. Could Search on Registration District be made available as it is currently on pilot?

    2. The ability to search on marriages so as to be able to locate the other party to the marriage. This should be possible since Year, Volume nr and page number were all transcribed during the transcriotion project a couple of years ago.

    3, Similarly the ability to search for children of the same parents.

    I know it is probably very different but the search capabilities are already available on FreeBMD and the Irish Civil Registration Index is almost identical to the England & Wales upon which it is based.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Thank you Robert,
    The lack of filtering ability drives me crazy, but I have still managed to find some wonderful information on the beta site. I'm willing to wait for a quality product.
    I started genealogy before computers so can remember the good old days. Computers are easier. I appreciate all the work your staff does.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • Please don't eliminate the Pilot program because I was able to get 1860 Lawrence Co., Mississippi Census on pilot that I can't get on Beta unless I pay a fee. I like Beta after I've used Pilot EXCEPT it doesn't limit to the USA even though I put United States into Beta.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Well, thank you Mr. Brimhall. That is the very first response I have read in all the postings that actually makes sense and explains a few things. AND, best of all it tells us that you are actually reading our feedback. I have sent dozens of feedback messages. Yes, some were just frustrated negatives. But, most were examples of how the Pilot site worked and what isn't working on Beta.
    I LOVE the Pilot site and it has helped my research so much I cannot even tell you. Please know that we are all out here with thanks in our heart that there is an organization such as family search that is providing this information to genealogists. We are a tenacious group. We spend lots of time visiting many websites and we know what works well and what doesn't. Most of us are extremely computer literate too.
    Please continue to keep us informed and I'll look forward to the bricks & mortar and the final paint job on Beta.
    Sincerely,
    D. Hall, Ramona, CA
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • I’m frustrated
    I should have read this first before my 1st message, I see now where you are heading but just seeing a index if you need more clarification does not quite do it. Such as Lenora Gore in 1860 in Georgia was indexed Lenora Gove because some sweet person didn't know of the old ways of making R's.With out a image I would not have found her or her kids. Also I was able to get several Gore death certificates and when I went into show some one how good this was they were gone. Thus my frustration. Judy Gore Lent
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m happy
    Thank you for the update.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m Encouraged
    Thanks for an update on what is really happening.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m hopeful
    Please thank Robert Kehrer for his clarity and humour - we appreciated the kitchen comparison - my husband says we will settle for stainless steel fittings!

    I haven't used either Pilot or Beta that much because there was little on Pilot of relevance to my genealogy and Beta's lack of tight search parameters irritated me too much! However, I've tried to check developments at intervals as I spend a lot of my time teaching/helping others with FH and need to know what is available.

    Robert asks for constructive ideas:
    1. Easy access to a list of what has been included - years for each parish, not a blanket statement of "parish registers for Cheshire" - preferably with a rough % of the total or something like '23 parishes out of the total of around 140'.
    2. Some collections provided simply as a scrolling 'film' like viewing a microfilm at home. Yes, we'd have to use our eyes to search like in the old days, but on old films that often works better than guessing how a name may have been spelled by the creator or read by the indexer! For those of us living at great distances from the nearest FHC, the opportunity to access the films in this way would be a true blessing. I think the UK National Archives are offering this for some collections, so it must be technologically possible.
    3. A simple way to export large lists of results, so that they can be sorted, checked against gazetteers, investigated, and returned to over days, or longer.
    4. A clear source on each record - similar to "Extracted from Parish Register of St Nicholas, South Kelsey, Lincolnshire, England".

    For the two of us, www.familysearch.org was an almost perfect research tool - adding images, a few extra search abilities (wildcards, etc), and simple export of search result lists to .csv or other spreadsheet type format would have made it perfect!

    Many thanks for staying in touch and I will TRY not to get so irritated in future!
    • "Easy access to a list of what has been included - years for each parish, not a blanket statement of "parish registers for Cheshire" - preferably with a rough % of the total or something like '23 parishes out of the total of around 140'. "
      Absolutely - the current state suggests software designers simply don't understand sourcing. If I get a nil return looking for stuff in the parish of Nantwich, Cheshire - what does it mean? If I know all Nantwich PRs have been loaded, it means they weren't baptised there (subject to transcription errors). If I only know that 95% of Cheshire records have been loaded, what does that mean for me? Nantwich might be in the 5%...
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • I agree with Adrian that it would be very helpful if we could have an idea as to what is included in a database. If not that, could a note be added to each database as to the percentage completed? The pilot site, usually, told me what percentage of the records had been added.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I also think it would be helpful to know, at least, the percent completed for each database.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I totally agree, thank you. Still waiting for them to change that results page to a more user friendly format (like the pilot ) site.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m an earnest user
    Thank you, Cullen and Robert, for the information in this post and in the post where you posted further information. I now have a better understanding of the impetus for the shift in model and site from Pilot to Beta. I have to agree with much of the frustration over lost filtering capability, but trust that it will be re-engineered as soon as it becomes the highest priority in this HUGE project.

    I do have another concern, however. As an indexer, I have seen several projects pass through the indexing project and subsequently get listed as complete without then appearing as data in Pilot (now Beta). I understand there needs to be some lag time between the completion of the indexing and the availability as a searchable collection, and I also now understand some of the less-visible issues (like Pilot's technology issues and the licensing issues), but that does not, it seems, completely explain the collections which are not yet searchable. Since you ask for specific examples in feedback, one collection in question is Ontario Births 1869-1912.

    In response to other replies to this "management update", I agree with the suggestion that more information about the collection would at times be useful. One that was mentioned was the completeness information. I suspect that will be less of an issue as collections go from indexing to searchable. However, another item that could be of great interest and value would be a note as to why images are not available for certain collections. If it is because it is an old index which has not been reindexed yet with images, that is one useful datum; if the collection is available index-only because of licensing, it would be very helpful to know where to look for the images ("the images for this collection may be seen at www.commercial-site.com").

    Thanks again, for all the great work you have all put into this massive project.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • You technologists can only understand what we the genealogists need by searching your own family. The pilot site was perfect. The beta site isn't even close to being ok. I will try one search on the beta to see if it's better and when it isn't I go back to the pilot. Please change it back.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m frustrated
    Dear Technologists,
    How about giving all of your loyal users a Merry Christmas present and going back to the format of the pilot site. Wouldn't that just be the best present ever?
    Amen!
    D. Hall, CA
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m frustrated
    So far Mr. Brimhall, I am vastly unimpressed by the Beta site. It is harder to use and to find things (and yes, I have looked at the "what changes we made" information). You've managed to overcomplicate matters. It was much harder to find the "Historical Books" and using it was unsatisfactory as it brought up everything in the library catalogue and had to be winnowed to just "digital images." Even then it had to be further narrowed. I did not like it. It wasted my time, of which I value highly. I've read the complaints of other users and they are valid. You'd do much better to keep it simple.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated

  • Post Script: The digital image I called for would not load but came up "404" as it claimed it could not be found.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m confident
    Thank you Robert. I might be partial to the Pilot Project since I found out that my Grandmother had a shot-gun wedding there! (groom 43, salesman, bride 23, her father was a witness, there is no mention of a rifle...) I also added 4 twigs to my Swedish tree with a Michigan marriage record. Thanks for sharing the future plans with us
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • Cullen,

    Why is the England Marriages, 1538-1973 database no longer available on Record Search (Pilot)? I still prefer to use Pilot since it is so much easier to copy the source information, it is easier to review the results and gives more focused results. But it appears that Pilot is being crippled to force us to use what is now familysearch.org.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned happy, confident, thankful, excited kidding, amused, unsure, silly sad, anxious, confused, frustrated

  • I’m confused and frustated
    I don't think I am one of those people who hates change in general but I find the new site horrible after being so used to the old one. I know some change takes getting used to and I agree that the new site looks "slicker" but man, I can't easily do searches that I was able to do for years...and the results are much less user friendly as well. Please give us "old-timers" a choice!!

    Carl Lutes

    carl.lutes@gmail.com
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly happy, confident, thankful, excited indifferent, undecided, unconcerned sad, anxious, confused, frustrated