Q: Git: Recovering from re-written history. Merge conflicts I know, I know, re-writing history is terrible, but...
I accidentally checked in a large sql file to git that I needed to remove from all of history.
I clearly didn't do the right thing, but I ran something along the lines of:
git filter-branch --force --index-filter \
  'git rm --cached --ignore-unmatch dump.sql' \
  --prune-empty --tag-name-filter cat -- --all

It succeeded in removing the file, but caused things to be in a weird state:


*

*I ran the above command on the branch_a

*branch_b was untouched, and I

*tried to merge branch_a into branch_b

*I got a ton of conflicts, more than it felt like I should have gotten. But I fixed them, and
committed.

*I then made a change in branch_a, then tried to merge that
back into branch_b, but got all the same conflicts as the first time,
even thought I had merged them already.


Any idea how to resolve this? Did some dates get changed on commits causing this weird behavior?

A: OK, per comment, I think I can describe the problem properly now.
(This occurs with any kind of history rewrite, but if it's a non-shared repository, or all users of it are willing to update, it's possible to use git filter-branch successfully here.)
The starting point for what you need to know is that git never changes a commit, not for git rebase, not for git filter-branch, not even for git commit --amend.  What these commands do, in order to make things look changed, is make new commits, based on the old ones, but with something(s) changed.
Then, they move the labels around, so that you see the new commits instead of the old ones.

Here's a simple example of a repository with two branches and five total commits:
    B <- C   <-- br1
  /
A
  \
    D <- E   <-- br2

Commit A is the initial (root) commit and then there are two branch structures, consisting of commits B and C (labeled br1) and D and E (labeled br2).  (Each letter here represents some big hairy SHA-1 that is the "true name" of the commit, e.g., A might actually be 004257f....)
Each commit points back at its ancestor(s), so C points back at B, and B at A.  There are no "forward pointers": A does not point to either B or D.  The fact that B and D are both descendents of A must be computed, by starting at the end-points and working backwards.  (This is how commit D can be added without changing A.)
Next, you need to know that the word "branch" has two different meanings in git: there's what I call the "structural branches", where you can see that A <- B <- C  <-- br1 is one branch and A <- B <- C  <-- br2 is the other branch.  (The two branches share a common point at A, so you could say that br1 consists of just C and B, and br2 consists of just E and D.  But then which branch is A "on"?  It's really "on" both, in a sense, and if there were any commits before A they would also be "on both".  It's sometimes clearer to talk about branches "containing" commits, and there's a git branch --contains to show you the labels for branches that contain a given commit.)
Git's labels—things like branch and tag names, although "remote branches" are just labels as well, as are things like the stash reference for git stash, and so on—get things started: the labels are all find-able by name, and each one gives the raw SHA-1 for a commit1.  So br1 has the raw SHA-1 for commit C.  From there, git finds B and then A as well.  The branch label br2 finds E, which leads to D, and then to A.
The branch labels are also called "branch", although technically they only identify the tip of the branch.  That is, br1 makes C the "end" of a branch.
(Note that if we add a label pointing to B—say, add master—we've just added another branch.  This applies in both senses: there's a new branch label, master, and a new "structural branch", containing commits B and A.  It does not matter that this is overlapped by br1; it makes B a branch-tip-commit.  Or, we could make master point to C, so that branches br1 and master overlap exactly.  If we then add a commit F to br1, the branch-tip br1 moves forward to point to F but the branch-tip master remains behind, pointing to C.  This automatic advancement of the branch-tip is what it means to "be on a branch".  You can only be on one branch label, even if many labels all point to one commit, and then that's the branch label that advances.)
Now let's look at a simple case of a "history rewrite", where we get onto branch br2 and do git rebase br1.  The idea here is to pick up the parts of br2 that are not currently shared with br1—that would be commits D and E—and "move" them, resulting in:
    B - C           <-- br1
  /       \
A           D - E   <-- br2

(I've removed the arrow-heads from the internal commit links here only because they're hard to draw.)  But this is an in-accurate picture.  Git does not actually move the commits; this would require changing D.  The original commit D records its parent commit (or commits but here there's just one), and that's A.  Git can't change D—but it can copy it to a new commit, D', with everything the same except that D' says "my parent is C":
    B - C           <-- br1
  /       \
A           D'
  \
    D - E

Once that's in place, git can copy E to E', with everything about E' the same except (again) its parent: E''s parent is D':
    B - C           <-- br1
  /       \
A           D' - E'
  \
    D - E

All git needs to do now is move the label br2.  The label used to point to E; if it now points to E' instead, and nothing points to E, commits D and E become "abandoned" and can be garbage-collected later:
    B - C           <-- br1
  /       \
A           D' - E' <-- br2
  \
    D - E           [abandoned]


So, git filter-branch is basically "git rebase on steriods" as it were.  Instead of copying just a few commits, it copies all the commits, applying your supplied filters just before making each new commit.  Given our original simple A-through-E, if you do a git filter-branch --all, it copies every commit.
If your filters make no changes, the new copied commits are all completely identical (each file is the same, each tree is the same, and each commit is the same, so they all wind up using the same original SHA-1 IDs and none of the labels move either).  But in fact your filter must have made some change, probably fairly far back in history.  Let's assume in this case that it changes what is to go into commit A, so that git has to make a copy instead, A'.  The result of the filter-branch is this:
    B - C   <-- refs/original/heads/br1
  /
A
  \
    D - E   <-- refs/original/heads/br2

    B'- C'  <-- br1
  /
A'
  \
    D'- E'  <-- br2

(rather than just abandoning the previous branches, filter-branch leaves labels spelled refs/original/..., which you can delete if the result is what you wanted).
Note that it's copied everything from the point-of-change(s) forward.  But it's only done this in your local repository.  You mentioned that you had a "remote copy" (presumably something like origin).  They have only the originals, none of the copies.
Let's draw a more realistic commit graph, with o nodes for uninteresting points and a more complex history.  Let's assume further that the dump.sql file to be removed occurred before branches branch_a and branch_b split:
--o--W--X--Y--o--o--A   <-- branch_a
            \
             o--o--B    <-- branch_b

Here X is the first commit containing the unwanted file.  (We'll see why I labeled Wand Y soon.)  The filter-branch command therefore has to copy it, to X', and also all subsequent commits, including the two branch-tips A and B:
--o--W--X--Y--o--o--A    <-- [some label(s), incl refs/original/...]
      \     \
       \     o--o--B     <-- [some label(s)]
        \
         X'-Y'-o'-o'-A'  <-- branch_a
             \
              o'-o'-B'   <-- branch_b

Now, you said you "pulled the remote copy of branch_b".  That is, you did git checkout branch_a; git pull origin branch_b.
Here's where it gets a bit complicated. :-)
git pull is just git fetch followed by git merge.  The git fetch step goes to the named remote and brings over any new commits.  There were likely no new commits—you already have all the old ones, from which you made copies—but the merge step goes wrong here.
To account for this fetch, and/or an earlier fetch, where it says "some labels" above, let's add origin/ labels too.  There's an origin/branch_a pointing to commit A, and origin/branch_b pointing to B.  Again, these are the old commits.
So, git merge finds the merge-base for commits A' and B.  That's the point where their history forked, which is the commit just before X, i.e., commit W.  It then does a merge of A' against merge-base W and "their" branch B, by seeing what changed between W and B, and what changed between W and A'.
Whatever else it does, this will bring back the unwanted dump.sql, because that was added in going from W to B, and not added in going from W to A'.  It also picks up the two commits below where I'll replace o with *.  So the merged tree at this point has the new dump.sql in it, and you get a merge-commit tying branch_a to origin/branch_b:
--o--W--X--Y--o--o--A     <-- origin/branch_a
      \     \
       \     *--*--B      <-- origin/branch_b
        \           `--.
         X'-Y'-o'-o'-A'-M <-- branch_a
             \
              *'-*'-B'    <-- branch_b

Now let's say you ask to merge branch_a into branch_b at this point:
git checkout branch_b; git merge branch_a

Git again needs to find a merge-base.  (Or, if there are criss-cross merges between branch_a and branch_b, using the recursive strategy, it has to create a "virtual merge base" by combining equally-good ancestors, but let's keep it simple.)  In this case, the merge-base is Y'.  We'll take the changes since Y' from branch_b, i.e., the copies of the * commits, plus B'.  But we already have those changes because we merged origin/branch_b, which has the same changes (plus dump.sql).
Often this will "just work", but presumably in your case things were more complex and you got a bunch of conflicts.
In any case, things are going wrong because you're resurrecting the "old" (pre-"rewrite") branches by using the ones on origin.
This sort of thing is why it's difficult to "rewrite history" successfully: you can rewrite your own quite easily, but if you have published it elsewhere (e.g., on origin), other people have the earlier history, and you can't force them to get rid of it.  You have to get them to voluntarily give up the old "wrong" history and switch to the new "right" history.

1Or another git object, like an annotated tag; or a label can be an indirect reference, i.e., contain the name of another label.  Usually only the special HEAD label is an indirect reference, containing the name of the branch you're on.
