Current models propose that boundaries of mammalian topologically associating domains (TADs) arise from the ability of the CTCF protein to stop extrusion of chromatin loops by cohesin. While the orientation of CTCF motifs determines which pairs of CTCF sites preferentially stabilize loops, the molecular basis of this polarity remains unclear. By combining ChIP-seq and single molecule live imaging we report that CTCF positions cohesin, but does not control its overall binding dynamics on chromatin. Using an inducible complementation system, we find that CTCF mutants lacking the N-terminus cannot insulate TADs properly. Cohesin remains at CTCF sites in this mutant, albeit with reduced enrichment. Given the orientation of CTCF motifs presents the N-terminus towards cohesin as it translocates from the interior of TADs, these observations explain how the orientation of CTCF binding sites translates into genome folding patterns.