I’m frustrated

Arbitrator Quiz to ensure they understand Instructions and Updates before arbitrating that Project

Adding the Arbitration Agreement/My Results was a big step in making us aware of the mistakes that were being made. Is it possible to also implement a system wherein an arbitrator had to pass a proficiency quiz of sorts to demonstrate they have read and understand the Project Instructions and Project Updates before they can download a Project for arbing? It just seems like now that the indexers can see some of their mistakes, it would behoove the accuracy of the index to make sure arbitrators are following the same rules that indexers must follow. At the very least, maybe add a place for indexers to note why they captured what they did when clicking Feedback for review, as well as a place for Arbitrators to indicate whey they 'ruled' the way they did on the arbitration field in question?
59 people like
this idea
+1
Reply
  • Thank you for your suggestion. We are currently in the process of creating a self-assessment that allows individuals to see if they are able to successfully arbitrate.
    • view 4 more comments
    • Self assessment for many (or most) of these Arbs will be meaningless; in my time of Indexing, I have never seen such POOR and unqualified assessments of Indexed batches. These people do not bother to read the PI (look at the 1940 census as a MAJOR example -- the place /location columns where people lived in 1935 and how many Arbs added "same place or same house because it was written by the census taker, yet NOT to be included when indexing).

      There needs to be a testing or cleaning of house to remove these lackluster, unintelligent individuals who are doing nothing but making good Indexers want to quit in sheer frustration, as has been talked about on other forums. That, is the real shame.
    • Only FSI testing and certification could improve the arbitration accuracy problem. Self-assessment would be useful only for indexers.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • I’m frustrated
    5
    This is a great idea.I have been frustrated so many times by what I felt was bad arbing.By someone changing something that I knew was right.
    Kim
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 6
    I too have been totally frustrated with the way some of my batches have been arbitrated. When I began indexing I thought the arbs were tested and had to know all the rules for every project they worked on. I diligently read all instructions before beginning a new project and refer to them often so am very upset when an arb obviously hasn't. For the success of the records please change the procedures which allow anyone to qualify to become an arbitrator.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 7
    I think we need more than self assessment. Too many arbs think they are beyond needing to read rules and think because they have been doing it forever that the rules only apply to others. No amount of "self assessment" would help there.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • I’m frustrated
    1
    I had a personal experience with this.I was looking for my mom in law in the 1940 cencus and could not find her.Her name was Maudine King.After searching for several days I accidently came across her.They had her as Mandino King.On top of that on the same batch they had her mom Dannie Lee as Donnie Lee.I looked at that image and that was clearly an "a" in Dannie Lee.I had a lot of low scores because some of the arbs were not going by the same rules I am required to.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 8
    There needs to be an online test with instant feedback for each project. Demonstration of mastery would allow a person to arbitrate a specific project.

    Literally, the arbitrator would not see batches available for download except for projects they qualify to arbitrate.

    The online test could be used by indexers as well.

    Programming such tests would not be difficult.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 7
    I love the idea of a required assessment for arbitrators! So many times, pages that I have indexed have been arb'd incorrectly by someone who doesn't seem to know what the instructions are. For me, my score isn't what I am concerned about (although nobody like a low score.) My main concern as an indexer, and a researcher, is to make records accessible to the people looking for them.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • I’m sad and frustrated
    6
    Tests for arbitrators need to be manditory for each project, and repeated if new rules are added.
    Rules for each project need to be on one page and VERY easy to find.

    I had a batch the other day that a man was omitted completely during arbitration. He is no longer there!!! How will his family ever find him in those records!
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • I’m Speaking Out AGAIN!!
    4
    Well, those that know me know I have complained openly about several ARBS and while yes they are human and make mistakes, as handwritting is perception... I feel there should be a quiz given by Family Search for each project a person wishes to ARB.. Some just have BIG HEADS and its not about the info its how many DINGS they can get and then go gossip about it on other groups.... If you cant index it...you have ABSOLUTELY no business Arbing it and even then I think all rules should be gone over with others if u question yourself at all.... Its not a contest, its not a game and its not who is the best, its about the researcher and our ancestors and now that I have seen some of the arbs work, I understand why Ill never find my family.... Please, no matter how many "LIKES" this gets, it needs to be addressed as myself and some friends have considered giving up this effort several times due to lack of consideration on the part of FSI... But we have stuck it out and think you need to do the same for us... If this was a paid job, someone out there would sure make sure its done right, but in my opinion, volunteers deserve more respect as WE dont have to be here, we choose to be here.... HELP US , HELP YOU!!

    Tammy
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 6
    First, I wish there was some way for FSI to test arbitrators on how to arbitrate, i.e. beginning with record matching, which is essential on some projects. Many arbitrators either don't know how or don't want to bother with record matching, even though the tutorial stresses it and it's the first window that comes up after opening an arbitration batch. There's no excuse for an arbitrator to mark an indexer's work all wrong on say, lines 5- 40, because above line 5 somewhere that indexer just forgot one line or the arbitrator erroneously added a line. That happens because the arbitrator did not record match.

    Second, they should have a mandatory test on some of the Basic Indexing Guidelines, i.e. punctuation, how to enter names, dates, ages, etc. They should show they know where to find the Basic Indexing Guidelines for reference.

    Third, I agree with Dellory Matthews that each project should have a test on the PIs and PUs that arbitrators must pass to show that they read those instructions. I don't think that all indexers need a test...they'll learn through Arbitration Results eventually if the arbitrators are marking them correctly. But arbitrators, no matter how well-intentioned, are the quality assurance people here and need to know what they're doing.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 6
    I am an arbitrator and personally I want some sort of testing, although I agree self testing won't help or the "problem arbitrators" would have asked themselves a LONG time ago why they disagree with so many indexers. When I continually mark the same fields incorrect I always go back and check the project updates to see if something has changed that week. I review the field instructions and every project update at least weekly, often daily! I WISH there were clearer instructions for each project, but even similar projects will often have field helps worded in enough different ways to make total confusion from project to project. The basic rules for indexing should stay the same and EVERY approved arbitrator needs to know those rules, backwards and forwards before they even begin arbitrating.

    Not a day goes by that I haven't wished for some contact with someone at Family Search to tell me if my arbitration is okay. But I have never ever been contacted by anyone, all my training is self taught and the best training comes from reading Family Search forums at least weekly to check the latest posts.

    What protects the integrity of the Indexing program? I want to know that I'm making correct choices while arbitrating and positively aiding this great work, but where does that affirmation come from? I honestly don't know! As a long time family history researcher I know these concerns are valid and need to be addressed. Just one omitted name can create a road block and be the cause of great frustration for our beginner researchers.

    We need accountability!

    Deanne
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 5
    What's the worst thing that could happen if arbitrators were tested and certified before cutting them loose? You know, I can't really think of anything negative. I guess it could slow down getting a project completed by a few days or so. Can anybody else think of any reason why we should not have better quality control and accountability?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 1
    That's exactly what needs to happen--if quality really matters over quantity.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • I think this is a fabulous idea. I am not sure I'm ready to make the move to arbitrating so a test would ensure I was competent before starting.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • A most recent example, too, of the sheer bufoonery of some of these Arbs:

    Recently, I have been indexing batches of US soldier enlistments.

    On one batch I was Indexing, there is a column on the indexing form to fill in the place (city or town) where the soldier enlisted or signed up. The batch I was doing was for soldiers who joined the army between 1815-1828.

    For one individual, it indicated that he signed up at the Union court house, in South Carolina. Anyone with half a brain can see that this recruitment went to the court house in the town of Union and enlisted at that building. So, in the place column, I typed Union, then in the state column, I added South Carolina.

    I checked the arbitrated batch later, and once again, some half baked Arb dinged this entry and changed the place name (town) to Union Court House.

    Really???

    On no map for South Carolina, is there a city or town that goes by the name of Union Court House, SC.

    This is nothing but a moronic, lack of paying attention by these Arbs.

    What is with these individuals?? Are they truly that stupid? Or do they just not care, and sit there, like a bunch of robotic monkeys, simply adding and changing information simply because something is indicated on the original document, even though the PI indicates otherwise?

    Something, on some level, needs to be done. Good ans solid work done by Indexers is being butchered and mauled by these people and that is very much a disservice to the FamilySearch site, as a whole.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 3
    Something does need to be done. The question is what?

    I think we've already got the basic framework in place. We have indexers, we have arbitrators, we have a way for indexers to flag arbitration mistakes. (Is this ever checked?) These ideas don't require changes to the user interface*, only to the back end:
    *(though it might help)

    (1) When an indexer flags a mistake, it should go back to arbitration, with a different arbitrator. Only fields that the indexer has flagged for consideration should appear as different, permitting the new arbitrator to jump right to the issue.

    (2) Arbitrators can be given tests in the style of "secret shoppers". In other words, you pass them the occasional batch that has been test proofed, and see how they do. Bad arbitrators could then be given feedback, or reported to the local director. (Surely these directors can't teach their helpers - or pull them - if they don't know there's a problem.) As a side benefit, after a while you wouldn't need to create test proofs - You just pass a batch to a couple of known great arbitrators, and see what they can agree on! Just turn it back around and now you've got a test batch.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • 3
    I'd just like to chime in here on this VERY important topic. As relatively new (but trying to make up for time) indexers, there are three adults in our house who are about to go back to Find A Grave with our free time. We're indexing 17th-century UK Parish Registers, and we take this difficult work very seriously. About half of the Arbs have been fine, but the other half. Phewwwww. Many of them either can't see well or can't read old handwriting. Many of them haven't read the rules (e.g., babies' or wives' surnames). Many of them have no clue how to add or remove a record in the middle of a batch. Heaven forbid that an indexer makes an error near the beginning. All of a sudden, you've got this Arb retyping another 60+ records and marking yours wrong -- all because they're now one record off in number. There should definitely be a test for reading different kinds of old handwriting and the various abbrevations and symbols used by the churches back then. What do you do with an Arb who can't read "ye" as "the" or "his wife"? It often takes us a long time to get used to a certain recorder's handwriting. Some of those capital letters can cause you to knock your head against the wall. Some Arb comes along and, in five minutes, marks down inanely stupid "corrections".

    For now, our answer to this problem is that we're going to stop looking at the Arbitration Results. This feels like an important thing to do right now, but we've got other volunteer projects clamouring for our attention, too. PLEASE, oh, please, give your indexers a break and improve the quality of your checkers.

    Thanks for letting me say this. It isn't fun around here like it used to be . . .
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly

  • I’m frustrated
    1
    My arbitration results score just dropped like crazy. I went to find out why, and discovered that I had made a very irritating mistake. I typed the entire page as written, and got the given name and surname box mixed up.

    I'm ticked at myself for doing this... but I'm angry at the arbitrator. Arbitrators CAN and SHOULD send back improperly indexed batches, such as this one. It should not have been arbitrated. "Also remember that arbitrators are not supposed to be a third indexer."

    I went through the results anyway, to see if there was anything I could learn. There were about a dozen discrepancies between what I put, and what was accepted. At least half of those, I was clearly right. The other half were debatable. But I can't even select "Feedback" on these, because I was clearly "wrong" too. (Not that "Feedback" does anything whatsoever)

    There really needs to be a better way to ensure competent arbitration.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated indifferent, undecided, unconcerned kidding, amused, unsure, silly