I’m excited to be attending my first RootsTech. I look forward to meeting in-person friends and colleagues. I will be presenting on Thursday morning, participating as a panelist Thursday afternoon, and spending a lot of time at the NGS booth in the Expo Hall during the whole event. I hope you will stop by and say, “Hello.”
Here is where you can find me:
Presentation: Intro to AI Genealogy: “Five Tools for Your AI Genealogy Toolbox”
8 AM MT, Thursday, In-Person, Ballroom A
Panel: “Ethics in the Family History Community: Town Hall Discussion”
4:30 PM MT, Thursday, In-Person, Ballroom G
Live Q&A: “Learn about the NGS AI Program”
Twice daily at NGS Booth: 10:45 AM MT and 2:30 PM MT, RootsTech Expo Hall: National Genealogical Society
I could not be more excited about sharing this announcement and learning with you. Registration opens Tuesday 20 February 2024, 1 PM ET, for “AI Genealogy Seminars: From Basics to Breakthroughs,” one of eleven virtual courses offered this summer by the National Genealogical Society’s GRIP Genealogical Institute (formerly the “Genealogical Research Institute of Pittsburgh”). I will be teaching ten sessions, from basics to breakthroughs in AI Genealogy, and I am humbled to be joined by seven distinguished colleagues who were students (though “fellow pioneers” would be more accurate, as I learned as much from them as they did from me) in my Level 1 and/or Level 2 NGS AI Genealogy courses last fall and this winter. More information is included below, and at the websites noted below.
Course: AI Genealogy Seminars: From Basics to Breakthroughs Coordinator: Steve Little Date: 23-28 June 2024 Venue: GRIP Virtual Session Registration Opens: 1 PM ET, Tue 20 Feb 2024
Description:
As the AI Program Director at the National Genealogical Society (“NGS”) and a pioneer in the field, Steve Little will navigate participants through the foundational concepts to the frontiers of AI Genealogy. His sessions will chart the evolution of AI Genealogy, from its early stages to predictive trends in 2024. Sessions will cover practical skills in prompt engineering, bot building and GPT customization, and the latest in AI Genealogy advancements. Steve’s comprehensive expertise will provide attendees with the tools to not only grasp AI basics but also to apply sophisticated AI strategies to their genealogical research. The AI Genealogy Seminars offer a unique opportunity to learn from the first-hand experiences of industry leaders during the initial year of large language model integration into genealogy. Their insights will shape the course content and provide a diverse perspective on the evolution of this field. Featured experts include Blaine Bettinger, Maureen Taylor, Judy Russell, Dana Leeds, Nicole Dyer, Mary Kircher Roddy, and Mark Thompson.
NGS online course “Empowering Genealogists with Artificial Intelligence – Level One” or commensurate experience (approx. 20+ hours of hands-on AI genealogy experience). The GRIP 2024 course “AI Genealogy Seminars” is NOT for a genealogist’s first experience with large language models or other AI tools; however, if a student has 20+ hands-on hours with ChatGPT Plus (GPT-4) when the course begins, then an updated refresher of the basics and intermediate aspects of AI Genealogy will ensure the student is up-to-speed to take advantage of the advanced topics and special content instructors.
About NGS’s GRIP Genealogy Institute: Formerly the “Genealogical Research Institute of Pittsburgh”, GRIP 2024 is “the” event for genealogists and family historians who want to develop their skills while meeting new friends in a collegial and collaborative community.
Open GeneaGPT (the community-built genealogy AI tool) has been updated from version 2 to version 3 as of Monday 22 January 2024. This is a significant update, transforming Open GeneaGPT into a smarter and more engaging companion for exploring family history, making it easier and more enjoyable for everyone. It brings changes like better conversations, more helpful suggestions for next steps, ensuring a friendly and insightful journey into the past.
OpenAI’s custom GPTs are amazing tools for creating specialized bots that can perform various tasks, such as extracting data; describing documents, images, records; generating reports, stories, images; and much, much more. These GPTs are tools for the way you work; you build the AI tool you need, or find one on the store shelf. I have been exploring genealogy prompt engineering over the past year, building genealogy GPTs since November, and I have learned a lot about how to create and use them effectively. Recently, OpenAI launched their GPT Store, where users can share their custom GPTs with other ChatGPT Plus subscribers. This is a great opportunity to discover new bots and learn from others. I have shared five or six of my custom genealogy GPTs in the store, which are related to family history research (more are being finished in the lab). Custom GPTs are available to ChatGPT Plus subscribers ($20/month) for no additional cost in the ChatGPT Store.
These are the kinds of genealogy AI tools that I teach students to create; no programming skills are necessary. ChatGPT can interview you, to ask you what kind of AI tool you’d like to create. Then I can show you how to fine-tune the tool to suit your exact genealogy workflow. It won’t fetch you a Snickers Bar or promise to solve a 120-year family history mystery, but within the realm of what large language models can do today, this one does okay.
If you’d like hands-on, step-by-step instruction on how to build your own custom GPTs and other specialized genealogy AI tools, your own flock of bots, the National Genealogical Society is now enrolling for Prompt Engineering and Specialized AI Tools for Genealogists, which starts at the end of January 2024; six hours of instruction over four weeks is just the start; also includes a collaborative study group, sharing successes and learning from failures. The first section of 50 seats sold-out in days, so a second section has been opened. Learn more here: https://www.ngsgenealogy.org/ai/
Jump straight the the growing list of Genealogy Bots here or access them at OpenAI Store by searching for bots with the term "genealogy." Custom GPTs are free for ChatGPT Plus subscribers ($20/month). I also teach genealogists and educators how to make their own custom genealogy GPTs, hands-on, step-by-step; enrolling now.
Friends, you may have heard the announcement that the OpenAI directory of custom GPTs is now being unrolled to ChatGPT Plus users. Custom GPTs represent an advancement in AI usefulness, marking a step towards more customizable and versatile AI tools. According to the OpenAI, these custom GPTs are described as a means to “create for a specific purpose,” highlighting their adaptability and user-centric design. Wharton professor Ethan Mollick in his article “Almost an Agent: What GPTs can do,” emphasizes the current state and future potential of these tools, noting, “GPTs show a near future where AIs can really start to act as agents.” This statement underscores the transitional nature of GPTs as a bridge between current AI capabilities and the more autonomous agents of the future. Simon Willison, in “Exploring GPTs: ChatGPT in a trench coat?” offers a practical perspective, stating, “The combination of features they provide can add up to some very interesting results.” His experience reflects the innovative possibilities that arise when various capabilities of GPTs are combined. Together, these insights from OpenAI, Mollick, and Willison paint a picture of GPTs as transformative tools, offering both a glimpse into the future of AI agents and a practical platform for current applications.
I’ve got four GPTs (now five) that I’m publicly testing (these are genealogy-related GPTs or genealogy-adjacent; another half-dozen others are still in the lab). I think of these GPTs as little AI tools that you can create, save, repeatedly re-use, and share. Custom GPTs, also referred to as bots, assistants, or agents (though these terms aren’t technically synonymous), represent a method to save a bundle of prompts, custom instructions, and abilities (image analysis, image generation, document reading, etc.) in a profile that you can use and share. For us as genealogists, this means that when we find a prompt, series of prompts, or set of custom instructions, to reliably accomplish a genealogically useful task, we can save that process as one of these GPTs; then, when we need to accomplish that task again, that tool, that bot, that GPT, is already in our AI toolbox.
Months ago, professional genealogist Yvette Hoitink created and shared the first genealogy GPT (to my knowledge), Dutch Genealogy Bot, available to ChatGPT Plus users at https://chat.openai.com/g/g-MMm3v0QX3-dutch-genealogy-bot. When asked for a short summary of its abilities, the bot replied: “As the Dutch Genealogy Bot, I specialize in guiding you through the process of researching Dutch ancestry, using resources and insights exclusively from Yvette Hoitink’s Dutch Genealogy website. I can provide detailed information and methodologies for tracing Dutch heritage, and direct you to specific articles on DutchGenealogy.nl for further guidance and authentic information.”
These four five little bots are my initial efforts, for example:
Genealogy Eyes Look at images, photos, and documents through the eyes of a family historian. Try it from your phone! Take a snapshot of a cemetery headstone, document/record, or anything else, and, using the official ChatGPT app, upload the image, say a little about the image and what you want, and click Send. https://chat.openai.com/g/g-gmIAn5mh6-seer-of-roots
Lingua Maven A linguistic expert, I combine dictionary precision, usage panel insights, and style guide expertise. My skills encompass a vast lexicon, dynamic thesaurus, and in-depth knowledge of language evolution, etymology, and dialects. I am a comprehensive resource for analysis and interpretation. https://chat.openai.com/g/g-Rxyt3Xww1-lingua-maven
Genealogy Summarizer Create useful summaries from texts, images, documents, photos, records, and more. This bot looks at what you give it, determines (as best it can) what it is, and suggests several ways to summarize the item. You can then ask follow questions about the item, or collect all the suggested summaries. https://chat.openai.com/g/g-Kg79HuRVD-genealogy-summarizer
Sam the Digital Archivist Kinda like a spicy librarian. Open GeneaGPT’s over-caffeinated genealogy and family history friend. Embark on a journey through your past with our customized genealogist bot, designed to delve into your ancestry and lineage. Discover your roots while having fun and learning genealogical methods. https://chat.openai.com/g/g-v6WgbVnba-sam-the-digital-archivist
PS: Learning how to make these custom GPTs is a significant portion of the focus of the Empowering Genealogists series class, Level 2: Prompt Engineering and Specialized AI Tools for Genealogists, which starts at the end of January 2024: https://www.ngsgenealogy.org/ai/
If you’re curious about how artificial intelligence (AI) might help you become more efficient, consider experimenting with citations. The large language models (LLMs) that have become popular in the last year might prove useful in this area. I’ve been using ChatGPT to experiment with AI in the last month as a student in the National Genealogical Society (NGS) course, Empowering Genealogists with Artificial Intelligence taught by Steve Little. Steve has a great blog here: https://aigenealogyinsights.com/. In the NGS course, we had four sessions about the four main LLM transformations: summarize, extract, generate, and translate. Today I experimented with having ChatGPT help me generate citations.
Book Citation
When learning how to create citations, it’s a good idea to start with a basic book citation. The principles for citing books are carried over to citing other publications in genealogy research, like websites. For genealogy citations, we use the Evidence Explained (EE) style. This style is based on the Chicago Manual’s Humanities Style (as opposed to its author-date style). The format for a book citation is the same in both EE and Chicago.
I asked ChatGPT 4 to help me cite a book in Chicago format. It then provided me both the format for a bibliographic entry and the footnote/endnote. Here’s the full response:
To cite the book “Research Like a Pro: A Genealogist’s Guide” by Diana Elder and Nicole Dyer in Chicago format, you would generally use the following structure:
Bibliography Entry: Elder, Diana, and Nicole Dyer. Research Like a Pro: A Genealogist’s Guide. [Place of Publication]: [Publisher], [Year of Publication].
Footnote/Endnote:
Diana Elder and Nicole Dyer, Research Like a Pro: A Genealogist’s Guide ([Place of Publication]: [Publisher], [Year of Publication]), [Page number].
In these citations, replace [Place of Publication], [Publisher], [Year of Publication], and [Page number] with the appropriate information from the book. The page number is only necessary in the footnote or endnote and should correspond to the specific part of the book you are referencing.
It’s important to note that the title of the book is italicized in both the bibliography and the notes. Additionally, in the notes, only the first author’s last name is listed first, followed by the first name, while the second author’s name is in the standard format.
Next I asked it how to create a shortened version of the reference note for use as a subsequent note. Here’s the response:
For a shortened version of the reference note for “Research Like a Pro: A Genealogist’s Guide” by Diana Elder and Nicole Dyer, used in subsequent notes in Chicago format, you would generally include just the primary author’s last name, a shortened version of the title, and the page number being referenced. It would look like this:
Elder and Dyer, Research Like a Pro, [Page number].
For instance, if you are referencing page 45 in a subsequent note, it would be:
Elder and Dyer, Research Like a Pro, 45.
This shortened format is used after the first full citation of the work in your footnotes or endnotes to make subsequent references more concise.
ChatGPT did a great job with providing the accurate format for a Chicago book citation. The Chicago Manual of Style, 17th ed., gives the same format on p. 799, section 14.101.
Census Citation
I asked ChatGPT to create a census citation. I started with a citation that I had already made for myself for my ancestor, Eliza Ann Isenhour. I separated the citation I had previously made into multiple lines, removed the punctuation, added an extra word (webpage near accessed) and put the civil division into the same line as the county and state – just to see if it knew where to put that. Here’s my prompt:
You are an expert genealogist. Make a citation for Eliza Isenhour on the 1850 census following the Evidence Explained style manual used by genealogists. Here is the information about Eliza’s enumeration:
1850 U.S. census Polk, Montgomery County, Arkansas population schedule p. 405a dwelling 6 family 6 Eliza Ann Isenhour in Barnet Isenhour household database with images Ancestry https://www.ancestry.com/discoveryui-content/view/9812411:8054 webpage accessed 17 November 2023 citing NARA microfilm publication M432, roll 28
The response:
To create a citation for Eliza Isenhour in the 1850 U.S. Census following the Evidence Explained style, you would structure it like this:
“1850 U.S. census, Montgomery County, Arkansas, Polk, population schedule, p. 405a, dwelling 6, family 6, Eliza Ann Isenhour in Barnet Isenhour household; database with images, Ancestry (https://www.ancestry.com/discoveryui-content/view/9812411:8054 : accessed 17 November 2023); citing NARA microfilm publication M432, roll 28.”
This citation follows the general format recommended by the Evidence Explained style guide, which includes the year and type of record, geographic location, specific details of where the record is found (page, dwelling, family), the individual’s name and household context, the database and its URL, the date of access, and the specific archival reference (NARA microfilm publication and roll number).
I was pretty impressed! It got almost everything right. This citation is usable. It did not place Polk, the civil division, in the correct position according to Evidence Explained, 3rd ed., p. 237. The civil division should follow the words “population schedule.” I prompted ChatGPT to put Polk where it belongs, and it did. I then asked it to make a few other tweaks, like removing the a after 405a, which is the page number given in the source information from Ancestry, and asked it to add the word (stamped) after 405. Then I asked it to italicize the publication website, Ancestry.
It’s important to check for accuracy while ChatGPT is still evolving. Because LLMs aren’t search engines, they sometimes produce incorrect information, according to what it thinks you want to hear. When I asked ChatGPT to make a citation for Eliza Ann Isenhour on the 1850 census, without providing the details from her actual household, it suggested that I follow this format:
“1850 United States Federal Census, [State], [County], [Township or City], enumeration district [ED], page [Page Number], dwelling [Dwelling Number], family [Family Number], Eliza Isenhour; digital image, [Name of Database or Collection] ([URL of the database] : accessed [Date of Access]); citing National Archives and Records Administration microfilm [Microfilm Number].”
This is an okay suggestion, and I love that it created a template that you can fill in. However, it incorrectly tells you to include the enumeration district (ED). EDs were not used until the 1880 census. Read the full conversation here: https://chat.openai.com/share/b7638e29-d6f9-431f-85a9-d2a008ebeb7d
Why do LLMs produce incorrect suggestions? The GPT in ChatGPT stands for Generative Pre-trained Transformer. The AI has been trained by reading a lot of content and is best suited for helping you with creative tasks or helping you summarize a block of text that you provide. The way it works is by predicting what usually comes next when talking about specific topics. So, if you provide ChatGPT with the way you like your citations to look, then give it some information to use, it can probably help with that. While we’re still learning the capabilities of ChatGPT and other LLMs, it’s important to check their work. This will help you know if you can use it for the tasks that will help make your work more efficient.
Shortening Citations
Another usage you might try for ChatGPT and citations is asking it to shorten a citation you’ve already made, when you want to use it for a subsequent note. I returned to my first conversation with ChatGPT about creating a citation for Eliza Isenhour where I provided the details. After the full reference note citation was tweaked the way I like it, I asked it to shorten the citation for a subsequent note. Here’s the response:
For a subsequent note, a short form citation for Eliza Isenhour in the 1850 census can be created as follows:
“1850 U.S. census, Montgomery Co., Ark., Polk, p. 405 (stamped), Eliza Ann Isenhour.”
In this short form citation, the details are condensed, providing just enough information to identify the specific record without repeating all the information from the full citation. It includes the year, the abbreviated state and county, the civil division, page number, and the name of the individual.
My response:
Please include population schedule, abbreviated, and the dwelling and family numbers, which can also be abbreviated.
ChatGPT’s response:
Certainly! Here’s the revised short form citation including the requested elements:
“1850 U.S. census, Montgomery Co., Ark., pop. sch., Polk, p. 405 (stamped), dwl. 6, fam. 6, Eliza Ann Isenhour.”
In this version, “population schedule” is abbreviated as “pop. sch.,” and “dwelling” and “family” are abbreviated as “dwl.” and “fam.,” respectively. This format maintains the necessary details while being concise for a subsequent note.
This can be a starting point for helping you figure out how you want to shorten citations for subsequent notes in a particular piece of genealogical writing that you’re creating.
Conclusion
It seems possible that LLMs could help us start drafting our citations. However, without me providing the details for the 1850 census that I already knew would be helpful, it had a much harder time. Maybe we can train our chatbots to give us citations in the format we prefer after providing them with additional examples and training.
The book citation was great, so using ChatGPT for help citing common source types, like books and articles online, is a great usage right now.
Your Turn
If you experiment with asking ChatGPT to create a citation for you, share below in the comments how it turned out. Was the citation formatted the way you’d expect according to the style manual?
Note: If you are sharing any private information as you practice making citations, turn off your chat history so it’s not seen by the engineers training the chatbot. See https://help.openai.com/en/articles/7730893-data-controls-faq for more info.
More AI and Genealogy Resources
Bettinger, Blaine.”Unlocking Family Secrets with AI | Findmypast,” 22 March 2023. YouTube. https://www.youtube.com/watch?v=exepLKC72Ts. This webinar is a great introduction to LLMs and Artificial Intelligence for genealogy and covers the purpose of LLMs, what they can do, and what they can’t do.
——. “10 ChatGPT Prompts Every Genealogist Needs to Know | Findmypast,” 10 May 2023. YouTube. https://www.youtube.com/watch?v=EbRXzd2SmNM.This gives some great ideas for how to use ChatGPT in genealogy.
Little, Steve. “Empowering Genealogists with Artificial Intelligence 6 September 2023.” YouTube. https://www.youtube.com/watch?v=npQaRJbzE1s. This is an overview of AI for genealogy and some of the capabilities of ChatGPT.
ChatGPT with Vision (GPT-4V) analyzes a handwritten WWII draft card, and not only reads handwriting correctly but also accurately identifies text fields (name, address, next of kin, date of birth, occupation, etc.). This draft card, from my maternal grandfather Dean Lawrence (1921-2003) was chosen for the average block print handwriting; testing will continue to determine how good GPT-4V is with recognizing cursive handwriting, but it appears to handle average print well. Again, though, the innovation is that the AI will correctly label the data; that is, earlier OCR can process simple block handwriting, but it couldn’t tell you whether the text was name, address, or occupation. ChatGPT with Vision (GPT-4V) correctly labels the data. Now, imagine a simple Python script that processes a folder of 20 or 200 or 2000 images like this for you; that will be a trivial task once OpenAI makes API access available for GPT-4V, which is expected (more simply, that is, with middle-school programming skills, tasks such as these will be quickly and easily automated).
PROMPT: You are a professional genealogist; your area of expertise is documents and records; you are adept at extracting every piece of information from a document or record. You are also a skilled writer and image analyst. Think now about a World War II Draft Registration Card, also called DDS Form 1: list every conceivable piece of information that might be found on a draft card (focus especially on names, relationships, dates, places, events, and facts).
Then, find attached a draft card. Extract every piece of information from the image of the draft card.
Finally, create a biographical narrative report from the text on the draft card; dry recitation, no editorializing.
A couple of points to highlight:
you can assign multiple roles simultaneously to the AI: you can prompt the AI to act not only as an expert genealogist, but at the same time to act as a skilled writer and image analysist.
you can ask for several tasks, essentially stepping ChatGPT through a process.
you can ask for a narrative summary of the text in the image, e.g., “create a biographical narrative report from the text on the draft card; dry recitation, no editorializing.”
Here was the response:
RESPONSE:
Information Extracted from the Attached Draft Card: 1. Serial Number: 234 2. Full name of the registrant: Warren Dean Lawrence 3. Place of residence: West Jefferson, Ashe, NC 4. Mailing address: Same as residence 5. Telephone number: Not provided on this card 6. Age in years: 20 7. Date of Birth: May 21, 1921 8. Place of Birth: West Jefferson, USA 9. Name and address of someone who will always know the registrant's address: Mrs. Warren Dean Lawrence, West Jefferson, NC 10. Employer's name and address: Partially illegible but appears to reference a mining company 11. Place of employment or business: West Jefferson, Ashe, NC 12. Signature of the registrant: Appears to be "Dean Lawrence" or a similar variant 13. Date of registration: Not clearly visible on this card
Biographical Narrative Report: Warren Dean Lawrence, residing in West Jefferson, located in Ashe County, North Carolina, registered for the World War II draft. Born on May 21, 1921, in West Jefferson, USA, he was 20 years old at the time of registration. His mailing address was the same as his place of residence. The individual who would always be aware of Warren's address was Mrs. Warren Dean Lawrence, also of West Jefferson. Warren's employment details reference what appears to be a mining company located in West Jefferson, Ashe, North Carolina. The draft card carries what seems to be his signature, "Dean Lawrence" or a similar variant.
Prompt Chain to process data further
Once we have the “Information Extracted from the Attached Draft Card,” that information can be standardized, so that if you have many sources of this type (in this example, a draft card), the data can be collected, compiled, compared, and analyzed. For example, if you had 20 of these cards, you may wish to move that extracted data into a spreadsheet or database. Saving the data for each record as a CSV (Comma Separated File, a widely-used file format) file allows us to do this simple task. In the instance, I followed-up the first prompt with a second prompt; using the results of one task as the input for a subsequent task is called prompt chaining. Here was my second prompt:
PROMPT: Create a CSV file from the Information Extracted from the Attached Draft Card. Use the format: "FIELD NAME","VALUE", where FIELD NAME is the field label used in the original image and VALUE is the value extracted from the image, such as "NAME","Warren Dean Lawrence". Display the response in a code window for easy copy-and-paste.
ChatGPT responds with the data in the format of a CSV file, and places that information in a code window, which makes copying-and-pasting easy (just click the “📋 Copy code” icon). One beta mode of ChatGPT, called “Advanced Data Analysis” (formerly “Code Interpreter”), will create an actual CSV file and present you with a download link, but for now we can only use one beta feature at a time; expect that to get better in time.
Up Next: Clean, simple cursive handwriting on a draft card
There remain many document types to test. I’ve been testing more challenging samples to discover where the limit of ChatGPT’s handwriting recognition. Up next is a clean, simple cursive script on a form such as this draft card. Ultimately, however, we will test handwritten documents such as letters, diaries, journals, court orders, and probate files. Handwriting recognition is hard, so I expect the limit might be discovered sooner than anticipated, for the time being.
For the record, here is a screenshot of the beginning of this ChatGPT conversation:
Months of waiting came to an end on Tuesday 3 October when I finally got to test ChatGPT with Vision (GPT-4V). This version of ChatGPT can now “See, Hear, and Speak.” I spent a few hours getting acquainted with GPT-4V. This report provides a brief overview of my experience, though there’s much more to explore.
Introduction to GPT-4 with Vision and OCR
ChatGPT with Vision isn’t just your average virtual assistant. It can hold conversations, process vast amounts of information, and even boasts a robust Optical Character Recognition (OCR) feature. With these capabilities, I decided to explore the possibilities of extracting genealogical data from visual charts.
The initial test was to attempt to extract data from an image, specifically, to extract biographical data from a photo of a genealogical chart and to save that data (names, dates, places, relationships) in a format that would be useful to a genealogist, in this attempt, an Ahnentafel list (a simple list to track ancestors).
Initial Fan Chart (partial) Failure
My first trial was with an image of fan chart. As beautiful as these circular wonders are, the curved text became a challenge for our OCR endeavor. Though GPT-4V got much correct, the curved nature of the fan chart’s text near the center made it difficult for it to capture the names accurately.
The results showed promise, but were not immediately useful. And that is typical. I NEVER get a prompt perfect on my first attempt, and it often requires several iterations of prompt refinements to get the quality of result desired. You can see from the results below that the AI did fine with the text that wasn’t curved, but it had trouble with the curved text near the center of the fan chart.
I strongly suspect this failure could be fixed, but my interest last night was to quickly find a successful use case that worked on a first attempt. So I moved onto a more simple challenge: a screenshot of a pedigree chart.
Success: Pedigree Charts and the Ahnentafel System
Recognizing the limitations, I shifted focus to pedigree charts, which present data in a more linear fashion. I first had ChatGPT note the value of the Ahnentafel system, a numerical method to track ancestors; this review has the effect of giving the AI a reminder of how an Ahnentafel list might be composed. Using this system, we set out to capture data from a pedigree chart and format it in an Ahnentafel list.
Here is the prompt I used; with GPT-4V an image can also be uploaded with the prompt. The screenshot above was included with this prompt:
PROMPT: Okay, I've got a simpler chart. First, tell me what you know about the Ahnentafel naming system. Think, then, too, about how the data in an image of a pedigree chart could be extracted via OCR and placed and stored in an Ahnentafel list. Find the attached image of a pedigree chart, extract the names, dates, places, and relationships, and place and store them in a Ahnentafel list (plain text is fine).
I was very pleased with the response. No, that’s an understatement–I was blown away by the response, on a first attempt:
The good news: ChatGPT (GPT-4V) OCR can effectively interpret an image of a pedigree chart, extracting the data and storing it accurately in an Ahnentafel file while preserving the relationship information inherent in the pedigree chart. All details accurate; no hallucinations.
This is significant. Because it is a relatively trivial task to then convert an Ahnentafel file to a GEDCOM, database, spreadsheet, or text file, the information in the image is now almost ready for import into your genealogy program (RootsMagic, Family Tree Maker, Gramps, etc..), Excel or Google Sheets, GDAT, Word, or simple text editor.
Data Extraction On-the-Go, with Your Phone
What’s more, you can do this on your phone! Here, with my smartphone, I took a picture of my laptop screen while a pedigree chart was displayed; GPT-4V correctly extracted the names, dates, and relationships from the photo, and then quickly presented it in a loose narrative report. The AI even picked-up on (correctly) and commented about the possibility of pedigree collapse and/or multiple relationships. All details accurate; no hallucinations. (You can see the full-size image here.)
Next Steps: More Tests; Implications; Possibilities
Next on my list: images of charts on paper, and neatly handwritten pedigree charts, etc.
Last night’s demo or proof-of-concept of extracting and saving biographical data in a genealogy-friendly format which preserves relationship information (the Ahnentafel file) from a picture or screenshot also suggests both clear implications and coming possibilities. A clear implication is that it is now much easier to get information off a printed page and onto the computer in a way that is genealogically meaningful because of the preservation of relationship information (inherently, the pedigree chart depicts who are the parents of whom, and this is captured and saved). A coming possibility suggests itself when we remember that API access to GPT-4V is coming, which means that we will be able to build apps and tools that process folders of our saved images and photos, or perhaps ask an AI assistant to do that for us.
Setting aside future possibilities, there are exciting days coming up now as we test other image use cases and work out the solutions to limits such as encountered with the fan chart. And folks will immediately find helpful this use case of converting an image of of pedigree chart to an Ahnentafel file.
Update:
If you are a ChatGPT Plus user, here is how you will know that GPT-4V has been rolled-out to your account (a process that OpenAI has said will take a couple of weeks). On your computer, tablet, or smartphone, look for a new image/picture icon near your prompt window. Here is what it looks like on a computer:
Reintroduction with Restrictions: ChatGPT Browse with Bing returns with enhanced guardrails after initial misuse concerns.
Performance Trade-offs: The updated ChatGPT Browse with Bing has more limited capabilities, affecting its speed and efficiency.
AI Interaction Tips: Engaging with AI as if it were sentient might yield better results, though it’s symbolic speech.
It’s been a busy week in AI developments: the long-awaited ChatGPT model that can see, hear, and talk began to be rolled out this week (I’m still waiting); Amazon invested $4 billion in Anthropic, the company behind Claude, ChatGPT’s strongest rival; Meta/Facebook is launching AI assistants in its messaging apps WhatsApp, Messenger, and Instagram. And much more.
Lost for a bit in the wave of news was the return of full internet access for ChatGPT. Called ChatGPT Browse with Bing, we had full internet access for a few weeks earlier in the year, but that Beta feature was discontinued when too many folks started using the tool to scrape (copy) websites and access content behind paywalls. So OpenAI pulled the plug to reinforce their guardrails. And, boy, did they tighten things down.
For folks who had ChatGPT Browse with Bing access in the spring for those weeks, there is a noticeable drop in performance in the re-release of the Beta mode. In the spring, ChatGPT Browse with Bing allowed users to apply the full power of GPT-4 to access and process web pages. And it was very useful.
For one thing, live internet access for the chatbot means an earlier limit was overcome. With live internet access, a model can have access to information more current than its training data. That is, without internet access for most of this year, ChatGPT had no knowledge of events after September 2021 when its training ended.
That advantage and benefit is restored.
But at a cost.
ChatGPT Browse with Bing appears to have been somewhat lobotomized; it might now be operating on a fine-tuned GPT-4 model, which could explain some of its altered behavior.
First, it appears that summaries of webpages are limited to a few hundred words, about 500 tokens, probably an attempt at fair use. You can still quiz ChatGPT Browse with Bing about a webpage and eventually get the results you desire. But the process takes much longer now than it did in the spring.
Second, look at the conversation I had with ChatGPT Browse with Bing earlier today. I was able to have ChatGPT Browse with Bing successfully create a list of genealogical education events between October 2023 and through summer 2024. It got all the details correct. But it took 26 prompt refinements. Admittedly, that means it only took about five minutes to build the calendar of events. But in the spring, it would have taken much less time and effort. I suspect this extra work is related to the guardrails that were installed. (This is the only mode of ChatGPT that appears to be effected; that is, other flavors of GPT-4 remain as robust as ever.)
Third, Reddit user ry4ny speculates that OpenAI has adjusted the presence penalty and temperature settings. As ry4ny notes, these changes might occur once the browser feature is engaged, transitioning the model from a ‘normal’ chat mode to a more restricted browsing mode. They also conjecture that the model might be using specific restart texts, which could explain the consistently repetitive endings in its completions.
So, enjoy ChatGPT Browse with Bing, but know that you will need to keep working to get the results you need.
Having just said that, now is probably a good time for a couple of reminders:
just as writing means rewriting, so prompting means re-prompting (I NEVER get anything perfect on the first attempt);
the AI does not have feelings, so it won’t get frustrated if you ask it 26 times to try again–sometimes that’s what it takes; that is, don’t worry about exasperating the AI, reiterate as much as you need to get the results you want; and
weirdly, for reasons that may not be yet fully understood, although the AI is not alive, you get better results when you talk to the AI as if it were a person (they’re called chatbots for a reason).
So it’s okay to use anthropomorphic language with and about the AI. We just occasionally remind ourselves these are figures of speech.
[LANGUAGE NOTE: Anthropomorphism is a figure of speech. AIs are not sentient. They are not alive. They do not "see," they analyze images; they do not "hear," they process audio signals; they do not "think," they evaluate. But it is okay if we speak as if AIs did see, hear, and think. We use figures of speech to communicate better.]
