In Fremont, a 4.0 is annoying but a pretty small earthquake. There are several in the area every year. In the eastern US, 4.0 is quite a bit more significant, but still pretty small. The difference is the rocks and soils under the surface. The ground of California and the west, with younger rocks and soils and a lot more earthquakes breaking things up, is a lot more flexible than that of the east, which is older and more settled. For example, the 5.8 that happened in Virginia in 2011 (60 times a 4.0) caused only minor damage, but that damage was over a much wider area than a same-size quake in California would have--as much as 150 miles away. A 5.8 in Fremont would have caused minor damage in Fremont (stuff falling off shelves, maybe some damage to masonry), but 100 miles away, likely couldn't be felt at all.
--Snortybartfast