MsHawklet is right. Leave the eggs in there for a while. Eventually the parents will push the unhatched eggs out of the nest, then it'll be safe for you to remove them.
I had a pair of zebra finches several years ago that quickly became 7 (5 hatched eggs). This happened a couple of times with them.
I now have two more zebra finches. I didn't want them to breed this time though, so I got two males. They have a grand time together and I don't have to worry about nests or eggs or hatchlings.
By
01/31/2012
Leave the other two eggs in the nest box until Mom ignores them. If you take the eggs out now and Mom still thinks they might hatch, it can prompt her to lay more eggs right away. This is called "double clutching." Double clutching is not encouraged because it can cause the Mom hen to deplete her calcium.
I've bred large parrots and cockatiels. This is the voice of experience.