There are different types of carpet glue, so different methods to remove them, and all of them are lots of work. First try laying some soapy water in a small area, then scrub a little with the mop. If that doesn't take it up, you can try mineral spirits in a small area. If that doesn't work, a store like Home Depot carries glue remover for cement floors, but all glue removers are not created equal. A store employee can give you further advice and/or recommend a glue remover.
The process would will probably involve a lot of scraping, some heavy rubber gloves and lots of ventilation. I hope this helps and good luck.