Took 10.4 for a spin this morning and found it to be noticably improved from 10.3.1 in New Jersey neighborhood driving!
Constant mini-braking events almost completely gone, somewhat smoother control through left and right turns, slight reduction in the severity of the wide left/right turn problem, overall felt less jittery. The times I disengaged or reported problems seemed more like things that I would expect it to have trouble with at this stage, as opposed to before where I was seeing a lot of stuff that I thought we should be past by now.
I'll see how it does with more drives of course, but overall very pleased that driving with it was less exhausting!