Time series work has three stages: get the dates parsed, make the time index the index, and then resample. Almost every problem in this area is a failure at one of those three, usually the first.
Get them parsed, then indexed
import pandas as pd
raw = pd.DataFrame({
'when': ['2026-01-15 09:30', '2026-01-15 14:05', '2026-01-16 08:00'],
'reading': [21.5, 23.1, 19.8],
})
print(raw.dtypes.to_dict()) # 'when' is an object - a string
raw['when'] = pd.to_datetime(raw['when'])
print(raw.dtypes.to_dict()) # now datetime64
series = raw.set_index('when')['reading']
print(series)
print(type(series.index).__name__) # DatetimeIndex
Open this example in the editor and change it. Nothing you type is saved.
Nothing in this lesson works until the dtype is datetime64 and the index is a DatetimeIndex. A string column that looks like a date supports none of it.
Parsing ambiguous formats needs care rather than a guess:
import pandas as pd
dates = pd.Series(['01/02/2026', '15/03/2026', '2026-04-20'])
# A format that matches them all does not exist, so parse in passes.
uk = pd.to_datetime(dates, format='%d/%m/%Y', errors='coerce')
iso = pd.to_datetime(dates, format='%Y-%m-%d', errors='coerce')
combined = uk.fillna(iso)
print(combined.dt.strftime('%Y-%m-%d').tolist())
# format='mixed' exists and is slower and more forgiving - which means
# it can forgive something you wanted to hear about.
print(pd.to_datetime(dates, format='mixed', dayfirst=True)
.dt.strftime('%Y-%m-%d').tolist())
Open this example in the editor and change it. Nothing you type is saved.
01/02/2026 is 1 February in Britain and 2 January in the United States. If both formats appear in one file, parse each explicitly and combine; a single call with dayfirst=True silently picks one reading for every row.
The .dt accessor is where the components live:
import pandas as pd
s = pd.Series(pd.to_datetime(['2026-01-15 09:30', '2026-07-04 22:15']))
print(s.dt.year.tolist(), s.dt.month.tolist(), s.dt.day.tolist())
print(s.dt.hour.tolist(), s.dt.dayofweek.tolist()) # Monday is 0
print(s.dt.day_name().tolist())
print(s.dt.to_period('M').astype(str).tolist())
print(s.dt.strftime('%d %b %Y').tolist())
print((s.dt.dayofweek >= 5).tolist()) # weekend
Open this example in the editor and change it. Nothing you type is saved.
Selecting by time
import numpy as np
import pandas as pd
index = pd.date_range('2026-01-01', periods=90, freq='D')
series = pd.Series(np.random.default_rng(2).integers(100, 400, 90), index=index)
print(series.loc['2026-02'].shape) # a whole month, by string
print(series.loc['2026-02-01':'2026-02-07'].shape) # inclusive of both ends
print(series.loc[series.index.dayofweek < 5].shape) # weekdays only
# The first and last week, by label arithmetic. (Series.first and
# Series.last existed for this and were removed in pandas 3.0.)
start, end = series.index.min(), series.index.max()
print(series.loc[start:start + pd.Timedelta('6D')].shape)
print(series.loc[end - pd.Timedelta('6D'):end].shape)
Open this example in the editor and change it. Nothing you type is saved.
String slicing on a DatetimeIndex is partial-match: '2026-02' means the whole of February. Note that it is inclusive of the end, unlike positional slicing - which is right for dates and surprising the first time.
Resampling
resample is groupby for time. It takes a frequency and an aggregation.
import numpy as np
import pandas as pd
index = pd.date_range('2026-01-01', periods=90, freq='D')
daily = pd.Series(np.random.default_rng(2).integers(100, 400, 90), index=index, name='revenue')
print(daily.resample('W').sum().head(4)) # weekly, ending Sunday
print()
print(daily.resample('ME').sum()) # month end
print()
print(daily.resample('ME').agg(['sum', 'mean', 'max']).round(1))
print()
print(daily.resample('QE').sum())
Open this example in the editor and change it. Nothing you type is saved.
The frequency strings worth knowing: 'D' day, 'W' week, 'ME' month end, 'MS' month start, 'QE' quarter end, 'YE' year end, 'h' hour, 'min' minute. Recent pandas renamed 'M' to 'ME' and 'Q' to 'QE', which is the cause of most deprecation warnings in older notebooks.
Going the other way - up-sampling - creates rows that did not exist, so you must say what fills them:
import pandas as pd
sparse = pd.Series([10, 40], index=pd.to_datetime(['2026-01-01', '2026-01-04']))
print(sparse.resample('D').asfreq().tolist()) # NaN for the new rows
print(sparse.resample('D').ffill().tolist()) # carry the last value
print(sparse.resample('D').interpolate().tolist()) # straight line between
Open this example in the editor and change it. Nothing you type is saved.
Each is right for a different thing. A stock price carries forward; a temperature interpolates; a count of events should be zero, not carried:
import pandas as pd
events = pd.Series([3, 5], index=pd.to_datetime(['2026-01-01', '2026-01-04']))
print(events.resample('D').sum().tolist()) # 3 0 0 5 - correct for counts
print(events.resample('D').ffill().tolist()) # 3 3 3 5 - wrong for counts
Open this example in the editor and change it. Nothing you type is saved.
Rolling windows
import numpy as np
import pandas as pd
index = pd.date_range('2026-01-01', periods=60, freq='D')
daily = pd.Series(np.random.default_rng(6).integers(100, 400, 60), index=index)
print(daily.rolling(7).mean().head(8).round(1).tolist())
print(daily.rolling(7, min_periods=1).mean().head(3).round(1).tolist())
print(daily.rolling('7D').mean().head(3).round(1).tolist()) # time-based
print(daily.expanding().mean().head(3).round(1).tolist())
print(daily.ewm(span=7).mean().head(3).round(1).tolist())
Open this example in the editor and change it. Nothing you type is saved.
rolling(7) means seven rows; rolling('7D') means seven days, which is different as soon as the series has gaps. For irregular data the time-based form is the correct one.
A rolling window ends at the current row by default, so it uses no future information. That is what you want for anything that will run on live data. center=True puts the row in the middle, which is fine for a smoothed chart of history and wrong for a feature in a model.
Time zones
import pandas as pd
naive = pd.to_datetime(['2026-06-15 12:00', '2026-01-15 12:00'])
print(naive.tz) # None - no zone at all
utc = naive.tz_localize('UTC') # ATTACH a zone
london = utc.tz_convert('Europe/London') # CONVERT to another
print(utc.strftime('%Y-%m-%d %H:%M %Z').tolist())
print(london.strftime('%Y-%m-%d %H:%M %Z').tolist())
Open this example in the editor and change it. Nothing you type is saved.
The two verbs are not interchangeable. tz_localize says "these numbers were always in this zone"; tz_convert says "show me this instant in another zone". Localising a series that is already aware raises, and converting a naive one raises too - which is pandas making you say which you meant.
Daylight saving is where naive timestamps become wrong:
import pandas as pd
# The hour that does not exist, in London, in 2026.
try:
pd.to_datetime(['2026-03-29 01:30']).tz_localize('Europe/London')
except Exception as error:
print(type(error).__name__)
# And the hour that happens twice, in October.
try:
pd.to_datetime(['2026-10-25 01:30']).tz_localize('Europe/London')
except Exception as error:
print(type(error).__name__)
# Both have an explicit answer, which you must choose.
print(pd.to_datetime(['2026-03-29 01:30'])
.tz_localize('Europe/London', nonexistent='shift_forward')
.strftime('%H:%M %Z').tolist())
print(pd.to_datetime(['2026-10-25 01:30'])
.tz_localize('Europe/London', ambiguous=False)
.strftime('%H:%M %Z').tolist())
Open this example in the editor and change it. Nothing you type is saved.
The rule that avoids all of this: store and compute in UTC, convert to a local zone only for display. A pipeline that does arithmetic on local times will be wrong twice a year, and both times the error is an hour.
Gaps and alignment
import numpy as np
import pandas as pd
a = pd.Series([1, 2, 3], index=pd.to_datetime(['2026-01-01', '2026-01-02', '2026-01-03']))
b = pd.Series([10, 30], index=pd.to_datetime(['2026-01-01', '2026-01-03']))
print((a + b).tolist()) # aligned by timestamp: NaN in the gap
print(a.reindex(b.index).tolist()) # keep only b's timestamps
print(b.reindex(a.index).ffill().tolist())
# Finding the gaps, rather than discovering them later.
index = pd.to_datetime(['2026-01-01', '2026-01-02', '2026-01-05'])
present = pd.Series([1, 2, 3], index=index)
complete = pd.date_range(index.min(), index.max(), freq='D')
missing = complete.difference(present.index)
print('missing days:', missing.strftime('%Y-%m-%d').tolist())
Open this example in the editor and change it. Nothing you type is saved.
Checking for gaps before resampling is the step people skip. A weekly sum over a series with three missing days is a smaller number with no indication that anything is wrong.
A worked example
import numpy as np
import pandas as pd
rng = np.random.default_rng(seed=41)
# Half-hourly readings for 30 days, in UTC, from three sensors, with
# a deliberate outage so there is a gap to find.
stamps = pd.date_range('2026-03-01', periods=30 * 48, freq='30min', tz='UTC')
frames = []
for sensor, base in [('kitchen', 21.0), ('loft', 16.5), ('garage', 12.0)]:
daily_cycle = 3 * np.sin(np.linspace(0, 30 * 2 * np.pi, len(stamps)))
frames.append(pd.DataFrame({
'at': stamps,
'sensor': sensor,
'celsius': (base + daily_cycle + rng.normal(0, 0.6, len(stamps))).round(2),
}))
readings = pd.concat(frames, ignore_index=True)
outage = (
(readings['sensor'] == 'loft')
& readings['at'].between('2026-03-10', '2026-03-12 23:59', inclusive='both')
)
readings = readings[~outage].reset_index(drop=True)
print(f'{len(readings):,} readings, {readings["sensor"].nunique()} sensors')
print('range', readings['at'].min(), 'to', readings['at'].max())
print('tz ', readings['at'].dt.tz)
print()
# 1. Find the gaps BEFORE resampling.
expected = pd.date_range(readings['at'].min(), readings['at'].max(), freq='30min', tz='UTC')
for sensor, group in readings.groupby('sensor'):
missing = expected.difference(pd.DatetimeIndex(group['at']))
print(f'{sensor:<8} {len(group):>5} readings, {len(missing):>4} missing slots')
print()
# 2. Index by time, then resample per sensor.
wide = readings.pivot(index='at', columns='sensor', values='celsius')
print(wide.head(3).round(2))
print()
daily = wide.resample('D').agg(['mean', 'min', 'max']).round(2)
print(daily.head(3)[('kitchen', 'mean')].tolist(),
daily[('loft', 'mean')].isna().sum(), 'missing loft days')
print()
# 3. Rolling, time-based so the gap is handled honestly.
rolling = wide.rolling('24h', min_periods=24).mean().round(2)
print('24h rolling mean, around the outage:')
print(rolling.loc['2026-03-09 12:00':'2026-03-13 12:00':48].round(2))
print()
# 4. Convert to local time for display only.
local = wide.tz_convert('Europe/London')
print('first UTC ', wide.index[0])
print('first London', local.index[0], '- GMT, so the clock agrees')
print('last UTC ', wide.index[-1])
print('last London ', local.index[-1], '- BST, an hour ahead')
print('same instants:', bool((wide.index == local.index).all()))
print()
# 5. Hour-of-day profile, computed on LOCAL time because that is what
# a person's day is - this is the one place local time belongs.
profile = (local['kitchen'].groupby(local.index.hour).mean().round(2))
print('warmest local hour', int(profile.idxmax()), profile.max())
print('coolest local hour', int(profile.idxmin()), profile.min())
print()
# 6. Fill the outage explicitly, and keep a flag.
filled = wide.copy()
was_missing = filled['loft'].isna()
filled['loft'] = filled['loft'].interpolate(method='time', limit_direction='both')
print(f'{int(was_missing.sum())} loft readings interpolated')
print('any NaN left:', bool(filled.isna().any().any()))
print()
# 7. Weekly summary, from the filled series.
weekly = filled.resample('W').agg(['mean', 'count']).round(2)
print(weekly[[('kitchen', 'mean'), ('loft', 'mean'), ('garage', 'mean')]])
Open this example in the editor and change it. Nothing you type is saved.
Five decisions make that correct. Everything is stored in UTC and converted only for the hour-of-day profile. The gaps are found and counted before any resampling, so a smaller weekly number is explained rather than mysterious. The rolling window is time-based, so the outage does not quietly shift a seven-row window across three missing days. The interpolation is explicit and counted. And the local-time profile is the one place a local timestamp is used, because "what time of day is it warmest in this house" is genuinely a local-time question.
The two conversions worth memorising
import pandas as pd
# Naive -> aware: say what zone the numbers were always in.
naive = pd.to_datetime(['2026-06-15 12:00'])
aware = naive.tz_localize('Europe/London')
# Aware -> another zone: same instant, different clock.
print(aware.tz_convert('UTC').strftime('%H:%M %Z').tolist())
# Aware -> naive, for a system that cannot cope: convert FIRST.
print(aware.tz_convert('UTC').tz_localize(None).strftime('%H:%M').tolist())
Open this example in the editor and change it. Nothing you type is saved.
tz_localize(None) after a tz_convert('UTC') gives you UTC wall-clock numbers with no zone attached, which is what most databases and CSV exports actually want.
When it goes wrong
| Symptom | Cause |
|---|---|
.dt raised an AttributeError |
The column is still a string; to_datetime first |
resample raised |
The index is not a DatetimeIndex |
| A date is a month out | Ambiguous 01/02 format; parse explicitly |
| Weekly totals are too small | Missing days; check for gaps first |
| An hour is missing or doubled once a year | Daylight saving on naive timestamps; use UTC |
tz_localize raised on an aware series |
Use tz_convert instead |
| A rolling window spans a gap | Use the time-based form, rolling('7D') |
A deprecation warning about 'M' |
It is now 'ME'; 'MS' for month start |
A check you can run
import pandas as pd
index = pd.to_datetime(['2026-01-01', '2026-01-02', '2026-01-05', '2026-01-06'])
series = pd.Series([10, 10, 10, 10], index=index)
print(series.resample('W').sum().tolist())
complete = pd.date_range(index.min(), index.max(), freq='D')
print('missing:', complete.difference(index).strftime('%Y-%m-%d').tolist())
Open this example in the editor and change it. Nothing you type is saved.
The weekly total is 40 across what looks like a six-day span, and the second line says exactly which two days are absent.
Run those two lines before every resample. A total that is short because of missing data looks identical to a total that is short because business was quiet, and only the second line can tell them apart.