While perceptual information arrives in a more-or-less continuous manner over time, our mind apprehends coherent and bounded subsequences that have beginnings, middles and ends and feel extended over time. For example, speech unfolds continuously without pauses between words, yet we understand meaningful units, at multiple hierarchical level, such as phonemes, syllables, words, and sentences, and ‘hallucinate’ pauses at the rate of those perceived mental units. A core problem has been to understand how and why the continuous flow of experience is partitioned in this way. In this talk I will present studies in which we have used invasive and non-invasive electrophysiology and computational modelling in tasks involving artificial sequences and visual narratives to shed light into the computations and brain mechanism mediating segmentation and encoding of sequences with the larger goals of  understanding the building blocks of our temporal experience and why time feels the way it does, e.g., how we can apprehend, feel, and marvel at the temporal structure of music.