Site icon DNAeXplained – Genetic Genealogy

DNAPainter: Painting “Bucketed” Family Tree DNA Maternal and Paternal Family Finder Matches in One Fell Swoop

Roberta Estes

DNAPainter has done it again, providing genealogists with a wonderful tool that facilitates separating your matches into maternal and paternal categories so that they can be painted on the proper chromosome – in one fell swoop no less.

Of course, the entire purpose of painting your chromosomes is to identify segments that descend from specific ancestors in order to push those lines back further in time genealogically. Identifying segments, confirming and breaking down brick walls is the name of the game.

DNA Painter New Import Tool

The new DNAPainter tool relies on Family Tree DNA’s Phased Family Matching which assigns your matches to maternal and paternal buckets. On your match list, at the top, you’ll see the following which indicates how many matches you have in total and how many people are assigned to each bucket.

Note that these are individual matches, not total matching segments – that number would be higher.

In order for Family Tree DNA to create bucketed matches for you, you’ll need to:

Yes, it appears that Family Tree DNA is now using 4th cousins, not just third cousins and closer, which provides for additional bucketed matches.

How reliable is bucketing?

Quite. Occasionally one of two issues arise which becomes evident if you actually compare the matches’ segments to the parent with whom they are bucketed:

Getting Started

Get started by creating or uploading your tree at Family Tree DNA.

After uploading your GEDCOM file or creating your tree at Family Tree DNA, click on the “matches” icon at the top of the tree to link yourself and your relatives to their proper places on your tree. Your matches will show in the box below the helix icon.

I created an example “twin” for myself to use for teaching purposes by uploading a file from Ancestry, so I’m going to attach that person to my tree as my “Evil Twin.” (Under normal circumstances, I do not recommend uploading duplicate files of anyone.)

Just drag and drop the person on your match list on top of their place on the tree.

Here I am as my sister, Example Adoptee.

I’ve wished for a very, very long time that there was a way to obtain a list of segment matches sorted by maternal and paternal bucket without having to perform spreadsheet gymnastics, and now there is, at DNAPainter.

DNAPainter does the heavy-lifting so you don’t have to.

What Does DNAPainter Do with Bucketed Matches?

When you are finished uploading two files at DNAPainter, you’ll have:

I’ll walk you through the painting process step by step.

First, you need to be sure your relatives are connected to your tree at Family Tree DNA so that you have matches assigned to your maternal and paternal buckets. The more relatives you connect, per the instructions in the previous section, the more matching people will be able to be placed into maternal or paternal buckets.

Painting Bucketed Matches at DNAPainter

I wrote basic articles about how to use DNAPainter here. If you’re unfamiliar with how to use DNAPainter or it’s new to you, now would be a good time to read those articles. This next section assumes that you’re using DNAPainter. If not, go ahead, register, and set up a profile. One profile is free for everyone, but multiple profiles require a subscription.

First, make a duplicate of the profile that you’re working with. This DNAPainter upload tool is in beta.

Since I’m teaching and experimenting, I am using a fresh, new profile for this experiment. If it works successfully, I’ll duplicate my working profile, just in case something goes wrong or doesn’t generate the results I expect, and repeat these steps there.

Second, at Family Tree DNA, Download a fresh copy of your complete matching segment file. This “Download Segments” link is found at the top right of the chromosome browser page.

Third, download your matches at the bottom left of the actual matches page. This file hold information about your matches, such as which ones are bucketed, but no segment information. That’s in the other file.

Name both of these files something you can easily identify and that tells them apart. I called the first one “Segments” in front of the file name and the second one “Matches” in front of the file name.

Fourth, at DNAPainter, you’ll need to import your entire downloaded segment file that you just downloaded from Family Tree DNA. I exclude segments under 7cM because they are about 50% identical by chance.

click to enlarge

Select the segment file you just named and click on import.

At this point, your chromosomes at DNAPainter will look like this, assuming you’re using a new profile with nothing else painted.

Let’s expand chromosome 1 and see what it looks like.

Note that all segments are painted over both chromosomes, meaning both the maternal and paternal copies of chromosome 1, partially shown above, because at this point, DNAPainter can’t tell which people match on the maternal and which people match on the paternal sides. The second “matches” file from Family Tree DNA has not yet been imported into DNAPainter, which tells DNAPainter which matches are on the maternal and which are on the paternal chromosomes.

If you’re not working with a new profile, then you’ll also see the segments you’ve already painted. DNAPainter attempts to NOT paint segments that appear to have previously been painted.

Fifth, at DNAPainter, click on the “Import mat/pat info from ftDNA” link on the left which will provide you with a page to import the matches file information. This is the file that has maternal and paternal sides specified for bucketed matches. DNAPainter needs both the segment file, which you already imported, and the matches file.

click to enlarge

After the second import, the “matches” file, my matches are magically redistributed onto their appropriate chromosomes based on the maternal and paternal bucketing information.

I love this tool!

At this point, you will have three groups of matches, assuming you have people assigned to your maternal and paternal buckets.

It’s Soup!!!

I’m so excited. Now my matches are divided into maternal and paternal chromosome groups.

Just so you know, I changed the colors of my legend at DNAPainter using “edit group,” because all three groups were shades of pink after the import and I wanted to be able to see the difference clearly.

Your Painted Chromosomes

Let’s take a look at what we have.

There’s still pink showing, meaning undetermined, which gets painted over both the maternal and paternal chromosomes, but there’s also a lot of magenta (maternal) and blue (paternal) showing now too as a result of bucketing.

Let’s look at chromosome 1.

This detail, which is actually a summary, shows that the bucketed maternal (magenta) and paternal (blue) matches have actually covered most of the chromosome. There are still a few areas without coverage, but not many.

For a genealogist, this is beautiful!!!

How many matches were painted?

Expanding chromosome 1, and scrolling to the maternal portion, I can now see that I have several painted maternal segments, and almost the entire chromosome is covered.

Here’s the exciting part!

I starred the relatives I know, on the painting, above and on the pedigree chart, below. The green group descends through Hiram Ferverda and Eva Miller, the yellow group through Antoine Lore and Rachel Hill. The blue group is Acadian, upstream of Antoine Lore.

Those ancestors are shown by star color on my pedigree chart.

I can now focus on the genealogies of the other unstarred people to see if their genealogy can push those segments back further in time to older ancestors.

On my Dad’s side, the first part of chromosome 1 is equally as exciting.

The yellow star only pushed this triangulated group back only to my grandparents, but the green star is from a cousin descended from my great-grandparents. The red star matches are even more exciting, because my common ancestor with Lawson is my brick wall – Marcus Younger and his wife, Susanna, surname unknown, parents of Mary Younger.

I need to really focus hard on this cluster of 12 people because THEIR common ancestors in their trees may well provide the key I need to push back another generation – through the brick wall. That is, after all, the goal of genetic genealogy.

Woohoooo!

Manual Spreadsheet Compare

Because I decided to torture myself one mid-winter day, and night, I wanted to see how much difference there is between the bucketed matches that I just painted and actual matches that I’ve identified by downloading my parents’ segment match files and mine and comparing them manually against each other. I removed any matches in my file that were not matches to my parent, in addition to me, then painted the rest.

I’ll import the resulting manual spreadsheet into the same experimental DNAPainter profile so we can view matches that were NOT painted previously. DNAPainter does not paint matches previously painted, if it can tell the difference. Since both of these files are from downloads, without the name of the matches being in any way modified, DNAPainter should be able to recognize everyone and only paint new segment matches.

Please note here that the PERSON unquestionably belongs bucketed to the parental side in question, but not all SEGMENTS necessarily match you and your parent. Some will not, and those are the segments that I removed from my spreadsheet.

Here’s a made-up example where I’ve combined my matches and my mother’s matches in one spreadsheet in order to facilitate this comparison. I colored my Mom’s matches green so they are easy to see when comparing to my own, then sorting by the match name.

Person 1 matches me and Mom both, at 10 cM on chromosome 1. Person 1 is assigned to my maternal side due to the matches above 9 cM, the lowest threshold at Family Tree DNA for bucketing.

In this example, we can see that Person 1 matches me and Mom (colored green), both, on the segment on chromosome 1. That match, bracketed by red, is a valid, phased, match and should be painted.

However, Person 1 also matches me, but NOT Mom on chromosome 2. Because Person 1 is bucketed to mother, this segment on chromosome 2 will also be painted to my maternal chromosome 2 using the DNAPainter import. The only way to sort this out is to do the comparison manually.

The same holds true for the X match shown. The two segments shown in red should NOT be painted, but they will be unless you are willing to compare you and your parents’ matches manually, you will just have to evaluate segments individually when you see that you’re working in a cluster where matches have been assigned through the mass import tool.

If you choose to compare the spreadsheets manually to assure that you’re not painting segments like the red ones above, DNAPainter provides instructions for you to create your own mass upload template, which is what I did after removing any segment matches of people that were not “in common” between me and mother on the same chromosomal segment, like the red ones, above.

Please note that if you delete the erroneous segments and later reimport your bucketed matches, they will appear again. I’m more inclined to leave them, making a note.

I did not do a manual comparison of my father’s side of the tree after discovering just how little difference was found on my mother’s side, and how much effort was involved in the manual comparison.

Creating a Mass Upload Template and File

The instructions for creating your own mass upload file are provided by DNAPainter – please follow them exactly.

In my case, after doing the manual spreadsheet compare with my mother, only a total of 18 new segments were imported that were not previously identified by bucketing.

Three of those segments were over 15cM, but the rest were smaller. I expected there would be more. Family Tree DNA is clearly doing a great job with maternal and paternal bucketing assignments, but they can’t do it without known relatives that have also tested and are linked to your tree. The very small discrepancy is likely due to matches with cousins that I have not been able to link on my tree.

The great news is that because DNAPainter recognizes already-painted segments, I can repeat this anytime and just paint the new segments, without worrying about duplicates.

I did not keep track of how many segments I deleted that would have erroneously been painted. There were certainly more than 18, but not an overwhelming number. Enough though to let me know to be careful and confirm the segment match individually before using any of the mass uploaded matches for hypothesis or conclusions.

Given that this experiment went well, I created a copy of my “real” profile in order to do the same import and see what discoveries are waiting!

Before and After

Before I did the imports into my “real” file (after making a copy, of course,) I had painted 82% of my DNA using 1700 segments. Of course, each one of those segments in my original profile is identified with an ancestor, even if they aren’t very far back in time.

Although I didn’t paint matches in common with my mother before this mass import, each of my matches in common with my mother are in common with one or the other of my maternal grandparents – and by using other known matches I can likely push the identity of those segments further back in time.

Status Percent Segments Painted
Before mass Phased Family Match bucketed import 82 1700
After mass Phased Family Match bucketed import 88 7123
After additional manual matches with my mother added 88 7141

While I did receive 18 additional matching segments by utilizing the manually intensive spreadsheet matching and removal process, I did not receive enough more matches to justify the hours and hours of work. I won’t be doing that anymore with Family Tree DNA files since they have so graciously provided bucketing and DNAPainter can leverage that functionality.

Those hours will be much better spent focusing on unraveling the ancestors whose stories are told in clusters of triangulated matches.

I Love The Import Tool, But It’s Not Perfect

Keep in mind that the X chromosome needs a match of approximately twice the size of a regular chromosome to be as reliable. In other words, a 14 cM threshold for the X chromosome is roughly equivalent to a 7 cM match for any other chromosome. Said another way, a 7 cM match on the X is about equal to a 3.5 cM match on any other chromosome.

X matches are not created equal.

The SNP density on the X chromosome is about half that of the other chromosomes, making it virtually impossible to use the same matching criteria. I don’t encourage using matches of less than 500 SNPs unless you know you’re in a triangulated group and WITH at least a few larger, proven matches on that segment of the X chromosome.

Having said that, X matches, due to their unique inheritance path can persist for many generations and be extremely useful. You can read about working with the X chromosome here and here.

I noticed when I was comparing segments in the manual spreadsheet that I had to remove many X matches with people who had identical matches on other chromosomes with me and my mother. In other words, just because they matched my mother and me exactly on one chromosome, that phasing did not, by default, extend to matching on other segments.

I checked my manually curated file and discovered that I had a total of seven X matches that should have been, and were, painted because they matched me and Mom both.

However, there were many that didn’t match me and Mom both, matching only me, that were painted because that person was bucketed (assigned) to my maternal side because a different segment phased to mother correctly.

On the X chromosome, here’s what happened.

You can see that a lot more than 7 bright red matches were painted – 26 more to be exact. That’s because if an individual is bucketed on your maternal or paternal side, it’s presumed that all of the matching segments come from the same ancestor and are legitimate, meaning identical by descent and not by chance. They aren’t. Every single segment has an inheritance path and story of its own – and just because one segment triangulates does NOT mean that other segments that match that person will triangulate as well.

The X chromosome is the worst case scenario of course, because these 7 cM segments are actually as reliable as roughly 3.5 cM segments on any other chromosome, which is to say that more than 50% of them will be incorrect. However, some will be accurate and those will match me and mother both. 21% of the X matches to people who phased and triangulated on other chromosomes were accurate – 79% were not. Thankfully, we have phasing, bucketing and tools like this to be able to tell the difference so we can utilize the 21% that are accurate. No one wants to throw the baby out with the bath water, nor do we want to chase after phantoms.

Keep in mind that Phased Family Matching, like any other tool, is just that, a tool and needs some level of critical analysis.

Every Segment Has Its Own Story

We know that every single DNA segment has an independent inheritance path and story of its own. (Yes, I’ve said that several time now because it’s critically important so that you don’t wind up barking up the wrong tree, literally, pardon the pun.)

In the graphic above of my painted X chromosome matches, only the six matches with green stars are on the hand-curated match list. One had already been painted previously. The balance of the bright red matches were a part of the mass import and need to be deleted. Additionally, one of the accurate matches did not upload for some reason, so I’ll add that one manually.

I suggest that you go ahead and paint your bucketed segments, but understand that you may have a red herring or two in your crop of painted segment matches.

As you begin to work with these clusters of matches, check your matching segments with your parents (or other family members who were used in bucketing) and make sure that all the segments that have been painted by bulk upload actually match on all of the same segments.

If you have a parent that tested, there is no need to see if you and your match match other relatives on that same side. If your match does not match you and your parent on some significant overlapping portion of that same segment, the match is invalid. DNA does not “skip generations.”

If you don’t have a parent that has tested, your known relatives are your salvation, and the key to bucketed matches.

The great news is that you can easily see that a bulk match was painted from the coloring of the batch import. As you discover the relevant genealogy and confirm that all segments actually match your parent (or another family member, if you don’t have parents to test,) move the matching person to the appropriately colored ancestral group.

I further recommend that you hand curate the X chromosome using a spreadsheet. The nature of the X makes depending on phased matching too risky, especially with a tool like DNAPainter that can’t differentiate between a legitimate and non-legitimate match. The X chromosome matches are extraordinarily valuable because they can be useful in ways that other chromosomes can’t be due to the X’s unique inheritance path.

What About You?

If you don’t have your DNA at Family Tree DNA and you have tested elsewhere, you can transfer your DNA file for free, allowing you to see your matches and use many of the Family Tree DNA tools. However, to access the chromosome browser, which you’ll need for DNA painting, you’ll need to purchase the unlock for $19, but that’s still a lot less than retesting.

Here are transfer instructions for transferring your DNA file from 23andMe, Ancestry or MyHeritage.

If you have not purchased a Family Finder test at Family Tree DNA and don’t have a DNA file to transfer, you can order a test here.

_____________________________________________________________

Disclosure

I receive a small contribution when you click on some of the links to vendors in my articles. This does NOT increase the price you pay but helps me to keep the lights on and this informational blog free for everyone. Please click on the links in the articles or to the vendors below if you are purchasing products or DNA testing.

Thank you so much.

DNA Purchases and Free Transfers

Genealogy Products and Services

Genealogy Research

Fun DNA Stuff

Exit mobile version