If there was oil paint sand it down, primer it and then use oil based paint or it will peel off or crack.
I agree with frugalsunnie that you should prime for sure. Where I differ is that the best primer to use is Zinser 123. Roughing it up first does help also.
I used Kilz once with disastrous results. So I am not a fan of that paint. Had to junk the table I used the Kilz on. I would visit your local paint store such as Lowes, Sherwin Williams, Home Depot, people who know paint and how best to use the product. They may recommend Kilz or something better but I don't believe you need to use that type of primer. I would scuff up the surface with some scotchbrite pads to give the latex primer some tooth to adhere to then use the latex top coat. After using the scotch brite pads be sure to clean off the dust residue with a cloth. Tack cloths work best.
Try Kilz or a similar quality primer, read the label in the store to be sure you're getting a primer that will work for your purpose.
You should be able to rough the gloss of the oil based paint by first washing with vinegar or TSP, then applying Kilz before you paint with the latex.
I've done it several times, even over varnish on plywood panels. It looks fine and doesn't peel if you follow the directions on the primer label.
Add your voice to the conversation. Click here to answer this question.