Once information has been collected during the gather phase, it must be analyzed. The reality is that the Gather and Analyze phases are often very intermingled. The researcher does micro-bounces back and forth between gathering and analyzing. They find a relevant record perform some analysis on it, decide whether there is anything that should be captured or if there should be any adjustments to their research plan, update their research log and move on. It is easy to tell when someone hits this phase – they look like the image below.

Photo by SAN DRINO

At times the analysis becomes more prolonged than these brief sessions interspersed throughout the gather phase. An individual pulls together all of the information they have gathered and analyzes it together to see what new clues, insights, learnings or conclusions can be gleaned. Whether it is a more formal analysis session or smaller analysis steps during the gather phase, the following questions are usually the focus of the anaylisis.

• Is this information helpful?
• Is this about my ancestor?
• Are these the same people?
• Does this conflict with other information?
• What information is missing and why?

One of the biggest challenges to the analysis phase is that it requires lots of knowledge and context to adequately perform an analysis. An individual often has to know the history, culture and jurisdictions of the places related to the information being analyzed. The individual has to understand the specifics of the particular artifact being analyzed – how, when, why and by whom it was created. One must have all known information about the person or family being researched. This is a lot of information to keep in context while performing an analysis. The complex context keeping problem is further exacerbated by common challenges with records that are hard to read because of fading, older or poor penmanship and different languages.

To overcome these challenges an individual uses a variety of different tools or seeks help from others with greater knowledge. It is common for an individual to plot out information on maps or use gazetteers. Timelines are often used to help identify unlikely differences between events. Family group sheets or other ways of depicting and comparing individuals, families and their relationships are brought to bear. Date calculators come in handy for calculating ages or specific dates of events. There are also many different forms that get used to try and capture, compare and analyze information.

While the context keeping concern is one of the main challenges in the Analyze phase, other challenges exist. For example, an extension of the context problem is that even after analysis has been performed it is common for an individual to need to refer back to earlier analysis efforts. It is extremely hard to keep all of the context brought to bear in the previous analysis. We’ve also observed that it is hard for an individual to do date calculations. Often one doesn’t know what information in a record is important or what it means so it is overlooked. The list of challenges goes on.

Several years ago I was involved with a study of how novices do family history. One of the interesting charts created as part of this study was an affinity diagram of all the challenges observed as novices attempted to do genealogy. When we had completed the diagram we had clustered together into related groupings all of the problems observed. While some of the problems were unique to novices, most of the problems observed were just inherent to the domain of genealogical research. After looking at the diagram for a while we decided to read through each of the nodes in the diagram and flag it if the node was an analysis step. We were surprised to discover that about 80% of the problems encountered were characterized as analysis. 80% of the challenges we observed while watching these novices do family history occurred during the analysis phase. Much of it was due to the complex context keeping conundrum described above.

As a result of this study we came to believe that there was a lack of meaningful analysis tools available to genealogists. We began a subsequent effort to prototype analysis tools. The first of these was an attempt to create a paper prototype of a tool that would help an ordinary person decide if two people (for example, an ancestor in his or her pedigree and an individual in a historical record) were the same. The tool basically helped the user compare relationships, events on a timeline and locations on a map for the persons in question. Our goal was to help an ordinary person make this determination within 30 seconds with high reliability. We created an initial mock-up and tested it with about five individuals. Images of these mockups follow. The initial test results were very good but unfortunately we were redirected onto higher priority efforts and have never returned to this effort.

Relationship Comparison

Timeline Comparison

Location Comparison (example 1)

Location Comparison (example 2)

It is my personal belief that next to availability of records, analysis tools (or the lack thereof) are the largest problem and opportunity in the domain of genealogy. What are your thoughts? Has this problem already been solved and I just don’t know it?

As many of you already know the Family Tree project has been retired from FamilySearch labs. Many features of this project have already been released in the new FamilySearch website.

Since the retirement many of you have responded with questions about those features you loved the best, asking when they may appear in the new FamilySearch website. So let’s go through the list of the features you mentioned and if they are in our present plans.

Here we go!!

Folders view

The folders view taught us a lot about the records combined into ancestors; how to best represent them and easily work with them. We also learned that to show such a simple view was not only helpful but a bit misleading. The Family Tree project showed the basic records but didn’t show any contributions that others had inserted into the record. Even though you didn’t see them, when the record was moved the inserted contributions went with it to the new ancestor.

We are currently working on a way to help you see all of the contributions and any records (Temple, PRF, AF, membership, etc.) in a complete and clear way. And still keep the easy way to move records to their right places. We believe you will like it even better than you did folders.

Children in combine records

Several have mentioned that they have used spouses and children shown in the folders view to help understand which record doesn’t belong and needs to be separated. In new FamilySearch we currently show spouses in combined records.

Because of your feedback, we are investigating getting children displayed in the combined records view as well.

Patron-contributed ordinances

This feature was put in to understand how patrons can help us reduce duplication of temple work by recording evidence that the work has been completed. It has been particularly helpful as we worked to resolve our combining limitations. In the next release of new FamilySearch we will be increasing the combine limit to a level that should resolve most issues.

Despite this, we recognize that there are still some ordinances that are not yet represented in the system. If you can believe it, some are still being transcribed from non-computerized records. We bring batches of them into the system regularly when they are available.

We know that telling others you know the work is already done sends a clue to them to not redo an ordinance. We have plans to add this into new FamilySearch.

User who has reserved the ordinance

This feature was put into new FamilySearch in our previous release. Any ordinances that are reserved now show who reserved them. Clicking their displayed name tells you any contact information they are willing to share. Thanks to your diligence, this has led to several helping others get the work done more quickly!

When the person reserved the ordinance

The value of knowing when a person reserved an ordinance was discovered through the Family Tree project. Today with all the reservations in the system, some reserved many years before new FamilySearch, it is difficult to always determine exactly when a reservation occurred. However, you’ve help us understand that this can be valuable for recent reservations.

We originally considered showing a date when we started showing who reserved the ordinance. At your request we will put renewed effort into seeing how this can be accomplished in new FamilySearch.

Conclusion

Thanks again for making the Family Tree project such a great learning experience.

Keep coming back to labs.familysearch.org regularly as we unveil some of our latest thoughts on how to present features discussed in this post and others. We value your feedback. You help us understand the path we should take to make a system that will work best.

Once a user has decided what to do and either formally or informally created a research objective and plan, they move to the Gather phase of the workflow. In theory the Gather phase is simply a matter of looking at all of the sources in the research plan to see if there is any relevant information. Simple. Right? Wrong.

Photo by Rob Young

The Gather phase is fraught with mishaps, peril and adventure. (Ok, not usually peril although I’m sure someone has a story to tell about this…) Consider what a user must go through to get access to source material. I don’t have enough time to give an exhaustive listing but for the purpose of explaining this phase we can generalize into the following categories:

• Stuff you can search online
• Stuff you can see on microfilm or microfiche
• Stuff you can get through the mail or via help requests
• Stuff in someone’s memory
• Stuff only Indiana Jones can find

One of the great blessings of the modern world we live in is the large amount of information that has been digitized, indexed and made searchable on the Internet. While the reality is that only a fraction of the genealogically significant records of the world exist in this state, there are some very good record sets for certain types of research. For example, the US Census records available through Ancestry.com, FamilySearch.org, Footnote and HeritageQuest are a tremendous help in doing research in the United States for certain time periods. It is really phenomenal that I can go to a website and type in information about the ancestor I’m looking for and often find them in a matter of minutes (or less). There are still however challenges inherent to this approach. Transcription errors and name variations, for example, can at times make it hard to find the person you’re looking for. While it is convenient to type in your search criteria and see results from multiple record sets, this approach makes it challenging to execute a ‘process of elimination’ style research strategy. The basic problem with most of the systems for searching records online is that it is often ambiguous what the scope of the records searched was and how reliable the sites efforts to deal with name variations, etc. were. It is also hard to remember what you’ve already searched for. “Did I already try Barns without the ‘e’?” “Have they added any new record sets since I searched for Deodat Brewster two months ago?” This makes it hard to go to your research log and cross off a specific record set from your search.

If the information is not available online then it may be available on microfilm or microfiche. The Family History Library with thousands of branch libraries (also known as Family History Centers) holds the largest collection of genealogical records available on film. Users typically start by searching the library catalog for records of interest. If they live close enough to the Family History Library to go there in person they will likely have a great experience with easy access to films. If you aren’t able to show up in the library in person then you can order these films for a small fee and they will be shipped to your local branch library. While this is better than not having access, ordering a film and then coming back in a few weeks when the film arrives and then sometimes discovering that it wasn’t the film you needed after all, isn’t such a great experience compared to finding ancestors online in minutes. In fact, we’ve discovered that the majority of potential users are willing to do an online search but will not go through the microfilm process even if it is extremely likely that the information they are looking for is on the film. The Internet has trained our society (for good or bad) to expect immediate access to information.

One of the great surprises for me as I began doing genealogy was to discover how much I enjoy getting a copy of a record through the mail. There are several variations of this process. The basic approach is to submit a request (phone call, letter, e-mail, etc.) for someone to look up a record for you and then pay for the copy and the postage. This is often an effective way to get something from an archive without having to travel there. Of course this is not the most efficient way to do family history. It has all of the attributes of the microfilm process except that because you’re not sitting in front of the film you lose lots of context like who else is listed in the record, etc. One of the great innovations in this area has been the advent of services like Findagrave. You submit a request for a photo of a headstone to the service. The request gets routed to volunteers that live close to that cemetery. They go take a picture of the headstone and upload it to the site for you.

Another place to get source material is out of your memory or the memories of other relatives. We tend to undervalue this information. For some reason we find someone’s memories that were written down on a piece of paper 100 years ago more credible than what a grandparent can tell us about a family. I understand that memories aren’t perfect nor are they objective. They are however rich and powerful. Anyone can interview someone about their family and most people are happy to talk about family memories. It can however be challenging to capture and preserve the information in such a way that it can be shared and used effectively as a source.

Last but not least, is the Indiana Jones approach to gathering information. This can involve trekking through the woods to forgotten cemeteries, bribing a priest, or poking around a cemetery at midnight. This can be really fun and rewarding. It can also be expensive and frustrating. When you pull an Indiana Jones you decide to travel to the motherland and find those ancestors. People tend to be very naïve about what it takes to do this. This isn’t just like taking a vacation. Many people spend a lot of money on a trip like this and just figure they will show up in Lower-Bristol and find their ancestors. Then they discover that the records are actually located somewhere else, the archive has weird hours, the priest won’t let them in or any number of frustrating deterrents to success. I’ve personally planned and gone on one of these adventures. It took substantial effort before hand to know where to go when I arrived. I did tons of research on the Internet to know where the records might be found and what times places were open. The information was very hard to find. My success was greatest when I could track down a local expert (like Sue at the Walworth County Genealogical Society) to guide me. The highlight of the trip for me was realizing that one of my ancestor’s headstones had been buried too deep and that some important information was to be had below the ground.

Each of these different approaches have their own unique challenges. They also have some challenges in common. It is hard for people to keep track of where they have already looked and what they did or didn’t find. People often find something and in their excitement forget to write down sufficient source information or make an adequate copy of the artifact. Gathering information can take time and money. Because it takes time people often have to stop and then pick back up where they left off. It takes a lot of effort to continue a search after putting it on the shelf for a while. There may political, religious or social obstacles to getting access to the information they’re looking for. Often people find information but don’t understand what it is or recognize other clues in the record beyond the specific item of interest.

There are many things that could be done to make this easier for people. Certainly the more records made available on line the better. Users need tools that automate the process of tracking what searches have been performed and what records have been searched. They need tools that automate the capture of information and citations. Tools that help users keep context during the search and that help them quickly regain context when coming back to the search. These tools need to help with online, microfilm, mail, memory and Indiana Jones style gathering of information.

Do you have ideas or frustrations about the Gather phase of the genealogical workflow? Have you seen examples of good tools to help? Let us know. Leave your comments below.

A couple of weeks ago we introduced a high-level Genealogical Workflow. This is the first in a multi-part series of articles breaking down the workflow into the next level of detail. Whether you’re starting to research your ancestors for the first time, coming back to it after an extended pause or actively seeking out your ancestors, part of your work process (whether conscious or unconscious) will be deciding what to do.

Photo by doug88888

What are some signs that users have hit the ‘Decide What to Do’ phase? Perhaps the easiest way to know if they are in this phase is to listen to the questions they are asking (either explicitly or implicitly). Here are some of the most common questions asked when users are trying to decide what to do:

• What do I already know?
• What sources do I already have?
• What do my relatives know?
• What has already been done?
• Where do I start?
• Who can help me?

To successfully make it out of the ‘Decide What to Do’ phase, users need to answer the questions they are posing and then establish a goal and a plan of attack. Students of genealogy tend to make this a very formal process setting a specific research objective, identifying a research strategy and making a list of specific record sets to search. This approach to research is consistent regardless of who is doing the research. The level of formality varies greatly depending on the skill level and the complexity of the objective in mind. Novice users may not know to apply the rigor to document the objective, strategy and lists of sources to check. Expert users may choose on a simple problem not to document these things because they are capable of keeping it in their heads and may not even recognize that they are doing it (kind of like showing up to work safely without remembering the drive).

Novice users are likely to need help understanding the basics. They need strong direction in knowing what to do, what questions to ask and where the best places are to start trying to find new information. A few years ago, FamilySearch prototyped a system that was successful at helping novice users through the ‘Decide What to Do’ phase. (Check out the slides and abstract from a presentation by Grant Skousen) We built a system that posed questions that novice users might want to answer. The questions became the research objective; the system selected a research strategy and relevant records to be searched; and then guided the users through the process. The prototype was so effective that complete novices had usually outgrown the handholding in the prototype by the end of an hour of usability testing. Oh – they also found some ancestors.

Even though more experienced users are more likely to know the questions to ask to determine what to do, they still need support in the process. If their research takes them to a new place or time period, they may need coaching on research strategies. They will need guidance on what record sets are available and access to historical information. They may want help surveying work that has already been done. If they are coming back to their research after a break they may need help knowing what they were doing the last time they were working on the objective. They may also benefit from having access to experts on topics of interest.

The features needed to support users in the ‘Decide What to Do’ phase can be broken down into Directive, Coaching and Supportive features. Directive features tell users what they should do and may actually constrain them to certain strategies or paths. Directive features are more effective for novice users. Coaching features make suggestions but don’t constrain users. These features are best for moderately skilled users or advanced users stepping into a new research area. Supportive features can be thought of as a reference desk. These are helpful for moderate and advanced skilled users. Here’s a rough whack at what some of these features might be. This is not a list of what we are building, just a quick brainstorm on my part of how this phase of the workflow might be addressed.

Directive Features

• Task suggestion (suggests a broad range of tasks designed to engage users and deliver a quick win)
• Tree analysis (to determine the best place to start – more tightly constrained than the tree report listed under the Coaching Features)
• Question/Objective setting (tell users what questions they should be asking and then set an objective for them when they choose a question)
• Automatically select and execute a research strategy for users
• Automatically select records to be searched

Coaching Features

• Tree report with recommendations (analyze a tree and suggest places to work)
• Contact an expert (contact an expert for consultation on a subject)
• Find others working on similar objectives (find others working in similar locations, time periods or on the same surnames)
• Help requests (think Find-a-grave – users can ask someone else to do a task for them)

Supportive Features

• Strategy reference (a reference of research strategies)
• Historical reference (historical background for different places, cultures and times)
• Record catalog (catalog of available record sets)
• Record collection descriptions (description of record sets)
• Where I left off (a snapshot of what users were doing the last time they worked on an objective)

I’m sure that there are more features than these to be considered. I’m also sure that there are some very good features that have already been built by various vendors that help with the ‘Decide What to Do’ phase.

We’d love your help. Tell us what we’re missing. Point us to good examples of systems that already do this well.

The time has come to retire the very popular Family Tree project from FamilySearch Labs. Those that have been following FamilySearch Labs for a while recognize that the roots to the Family Tree project go back to the Pedigree Viewer project (also another Labs favorite). The fruits of the Family Tree project have made their way into the production site, new.familysearch.org. Now that the key features of the Family Tree are incorporated into new FamilySearch the time has come to retire the project.

The Pedigree Viewer was the precursor to the Family Tree project and was first introduced on FamilySearch Labs in September of 2006. At the time it was a revolutionary twist in the way people viewed family trees. There was not another family tree viewer like it. The Pedigree Viewer pre-dated Geni.com and any number of dynamic pedigree viewing knock-offs. The concept was simple take a Google Maps approach to displaying a pedigree. Overcome the problems people have keeping context when looking at their family tree. Make it fun to use. By the end of March 2007 the Pedigree Viewer had served its purpose. We had learned a ton and raised the bar on family tree interaction.

We suspended work on the Pedigree Viewer to move on to other prototypes (like the Life Browser) and then a few months later officially started the Family Tree Project releasing the first version to labs in December of 2007. The Family Tree project was intended to take the key learnings of the Pedigree Viewer project and build out a new faster, richer, easier to use interface for new FamilySearch. After almost two years of development, In November of 2009 the first features from the Family Tree project were released in new FamilySearch.

Now that most of the features from the Family Tree project have made their way into new FamilySearch and others are well under way, it is time to retire the project. At the beginning of February you will no longer be able to access the Family Tree project from FamilySearch Labs. We will move it down to the Retired Projects section of the FamilySearch Labs home page. While we won’t be able to keep the project functional like we have the Life Browser and the Pedigree Viewer, we will provide images of the application as a memorial to the great work of the team (and to help us remember what worked and what didn’t work in the project).

So as we say goodbye to our old friend we also want to thank all of you who have taken it for a test drive, suffered through version after version after version and above all sent us your feedback. We’ve received feedback from thousands over the past few years telling us what works and doesn’t work. We still don’t think we have it perfect but it is so much better than it would have been thanks to your help. Thank you!

Those of you that follow the blog or have seen the FamilySearch beta know that we’ve been putting a lot of effort into rebuilding the search part of FamilySearch. Just for fun, we thought it might be interesting to show you a few of the behind the scenes folks that are doing a great job bringing you a screaming fast and accurate search experience. (I must apologize ahead of time for the low volume in the video. This was also an experiment with my new iPhone 3GS video capabilities.)

In my last blog post I asked you to share what your major obstacles are in doing family history. I received responses as comments on the blog post, comments on Facebook and @replies on Twitter. Many of the comments clustered together. Others were expressed only once. The list below is an attempt to quickly summarize the responses and indicate their frequency.

• So few records available online/Access to records (4)
• Lack of citations (3)
• Cousins don’t feel same urgency to share photos/Desire to share (2)
• Don’t know where to start/what’s already been done (2)
• Not enough time (2)
• Preservation of records – protecting from fires, floods, etc. (1)
• Poor PAF Import to FS.org (1)
• Poor equipment in local Family History Center (1)
• Temple too far away (1)
• Hard to index records in another language (1)
• Hard to interpret records in another language (1)
• New duplicates showing after others import GEDCOMs into new.FamilySearch.org (1)
• Cost prohibitive (1)
• Hard to organize stuff (1)
• Hard to sort out the good info from the bad info (1)

One of the key tools we use as we try to find solutions to some of these problems is a process called Contextual Inquiry and Contextual Design (CI/CD). One of the many tools that come from this CI/CD process is called a consolidated workflow. Consolidated workflows are created by observing people and carefully documenting the process they go through to get the job done. The results of the observation are then combined or consolidated into one consolidated workflow. The genealogical workflow below is a high-level representation of the workflow we have observed as we’ve watched people all over the world doing genealogy.

I thought it would be interesting to try to correlate the issues mentioned above to where they fit in the overall genealogical workflow. Not everything fit on the map so I dropped a couple of things that don’t fit in the workflow but do fit in other parts of the overall picture. The numbers in the red circles on the image below represent the frequency that you expressed an obstacle in this area of the workflow.

So I have two questions for you:

1. Does the genealogical workflow make sense to you?
2. After seeing this view of the genealogical workflow, do any more obstacles come to mind?

Updated Trees

• British Isles Peerage, Gentry and Colonial American Connections
• Canada: New Brunswick, Millville
• England London Residence of London
• Europe Royal and Noble Houses (predominately England and France)
• Europe Royal and Noble Houses (predominately German)
• Pacific Islands: French Polynesia: Atuona Island
• Scotland Fasti Ecclesiae Scoticanae
• Wales Medieval Records Primarily of Nobility and Gentry

New Trees from the Pacific Islands

• Pacific Islands: French Polynesia: Hanaiapa Valley, Hiva Oa island, Marquesas Islands:
  Hanaiapa Valley on Hiva Oa island includes the villages of Hanapaaoa and Hanateku’ua, and the valleys of Hanaiapa, Hanapaaoa, Hanau’i, Hanatku’ua and Hanamenu. The database was compiled by Daniel Longstaff and is extracted from civil records of Hanaiapa (marriages 1891-1902 and deaths 1891-1926). There are gaps in the years covered by the records.

• Pacific Islands: French Polynesia: Hekeani, Hiva Oa island, Marquesas Islands:
  Hekeani Valley on Hiva Oa island includes the villages of Haneupe, Hanahehe, and Moea, and the valleys of Hanaupe, Hekeane, Hanahehe, Hanatea, Moea, and Ooa. The database was compiled by Daniel Longstreet and is extracted from civil records of Hekeani (marriages 1882-1896 and deaths 1882-1930).

• Pacific Islands: French Polynesia: Bora Bora island, Society Islands:
  Bora Bora is located in the Leeward group of the Society Islands. Bora Bora island is located about 140 miles north of Papeete. The principal settlement on Bora Bora is Vaitape, on the western side of the island. The database was compiled by Daniel Longstaff and is extracted from civil records of Bora Bora (marriages 1892-1900, deaths 1892-1941).

• Pacific Islands: French Polynesia:  Fatu Hiva island, Marquesas Islands:
  Fatu Hiva is the southernmost of the Marquesas Islands. Fatu Hiva civil records include the villages of Omoa, Ouia, and Hana Vave. The database was compiled by Daniel Longstaff and is extracted from civil records of Fatu Hiva (marriages 1889-1892, deaths 1886-1952).

• Pacific Islands: French Polynesia: Hao atoll and Amanu atoll, Tuamotu Archipelago:
  Hao, or Haorangi, is a large coral atoll in the central part of the Tuamotu Archipelago. Hao is located 575 miles east of Tahiti and is 34.5 miles in length. Amanu, also called Timanu or Karere, is a smaller atoll about 11 miles north of Hao. The database was compiled by Daniel Longstaff and is extracted from civil records of Hao and Amanu (marriages 1886-1900, deaths 1886-1957) with additions from other records.

I know the topic of obstacles in family history and genealogy isn’t new. It seems to be a rite of passage for genealogy bloggers. I don’t know what you did over the holidays. Hopefully you spent a lot of time with family and friends like I did. I also spent some time asking myself questions like this:

“I’ve been working at FamilySearch for almost six years. Lot’s of new innovations have happened in that time. Have we made any progress at overcoming the major obstacles of family history?”

Anyone else spend time over the holidays on that one?

Before I give you my take on the answer to that question (the first one, not the second), I’m curious what people think the current major obstacles are. I asked this question earlier today on Twitter and Facebook and here is a sampling of the responses I’ve received so far:

I’m curious, what are the major obstacles you experience when you work on your family history? Leave your thoughts in the comments below.

We have recently updated and added some community trees.

Updated

• British Isles: Peerage, Gentry and Colonial American Connections – New records and source citations added.
• Canada: New Brunswick: Southampton: Millville – Haines and Palmer genealogies added and merged with the existing tree.  A recent meeting with the Fredericton Genealogical Society in New Brunswick created interest in doing a project for the surrounding parishes of Bright and Queensbury.
• Scotland: Fasti Ecclesiae Scoticanae – Existing data received a second check and duplicate records were merged.
• Wales: Records Primarily of the Nobility and Gentry – New records and source citations added.

New

• Canada: Manitoba: Winnipegosis – Tree of those that lived in Winnipegosis and their descendants as far as can be identified.
• United States, Oregon, Hood River – Includes information on deceased individuals from histories of Hood River County, Oregon, the Hood River County portion of the 1860-1900 Wasco County, Oregon Census, and 1910-1930 Hood River County, Oregon Census. Available marriage and death records as well as information from other sources are included.