Pandas and NumPy Interview Questions and Answers: 60 Questions from Easy to Hard
This guide is for Indian freshers and experienced data analysts and Python developers preparing for interviews that test pandas and NumPy, from a first analyst role at a Pune IT services firm to a senior data engineering position at a Bengaluru product company. It has 60 questions in three levels: easy questions on arrays, dtypes, broadcasting, Series and DataFrames, loc and iloc, filtering and missing values; medium questions on groupby, merges, reshaping, dates, cleaning and the SettingWithCopyWarning; and hard questions on memory layout, copy-on-write, time zones, large files and performance. Answers follow pandas 2.x and NumPy 1.26 and 2.x and name the version where behaviour changed. Type each example into a notebook, change the data, and explain the result aloud.
- 60 questions, 3 levels
- Updated
- CoreHRx Jobs
What pandas and NumPy interviews cover
- NumPy basics: arrays versus lists, dtypes, shape and reshape, broadcasting, vectorized operations, and when a slice is a view or a copy.
- pandas core: Series and DataFrame, the index and label alignment, loc and iloc, boolean filtering, sorting and value counts.
- Analysis: groupby with agg and transform, merge, join and concat with the right how= option, pivot_table and melt.
- Real data: missing values, duplicates, messy text and amounts, dates in DD/MM/YYYY format, time zones and resampling.
- Senior topics: copy-on-write in pandas 2.x, memory use, large CSV files, Parquet, and replacing apply and loops with vectorized code.
How to prepare as a fresher or an experienced candidate
- Freshers: take one public dataset, such as city-wise sales or exam results, and answer ten business questions with pandas, checking each total by hand on a small sample.
- Practise the core moves until they are automatic: read_csv, info, isna().sum(), loc with a condition, groupby with agg, merge with how=left, and pivot_table.
- Learn to predict shapes and row counts before running code: how many rows a merge returns, what shape a broadcast gives, and what a groupby index looks like.
- Experienced candidates: prepare stories of real problems, such as a merge that duplicated rows, a report that ran out of memory, or an apply you replaced with vectorized code.
- Know your versions: pandas 2.x changed several defaults, and NumPy 2 changed type promotion, so say which version a behaviour belongs to.
Common mistakes and interview-day tips
- Merging without checking key uniqueness and row counts, which silently inflates totals.
- Using and, or or a chained index like df[mask][col] = value instead of &, | and a single loc call.
- Filling every missing value with 0 or the overall mean without asking why the data is missing.
- Writing a Python loop or apply with axis=1 for logic that np.where, np.select or a groupby transform can do on whole columns.
- In a live coding round, say the expected shape or row count before you run each step, and check it after; interviewers value candidates who verify their own work.
Pandas and NumPy interview questions: Easy questions (freshers)
20 questions1. Why use a NumPy array instead of a Python list for numeric data?
A NumPy array stores values of one type in one contiguous block of memory, while a list stores pointers to separate Python objects. That lets NumPy run an operation over the whole block in compiled C loops, which is much faster for numeric work and uses far less memory. Operators also differ: a list times 2 repeats the list, while an array times 2 doubles every element. Lists remain the better choice for small or mixed data and for data you append to often, because np.append copies the whole array each time.
2. What is a dtype in NumPy and pandas, and why should you check it?
The dtype is the type that every element of an array or column shares, such as int64, float64, bool, datetime64 or object. It decides the memory per value, which operations are fast and what results look like. NumPy picks one dtype that fits all inputs, so mixing ints and floats gives float64, and mixing numbers with text gives a string type on which arithmetic fails. In pandas, a numeric column that loads as object usually means a stray value such as a dash or 1,200 with a comma; read_csv already treats NA, N/A and empty cells as missing. Check df.dtypes right after loading.
import io
import numpy as np
import pandas as pd
print(np.array([1, 2, 3]).dtype) # int64 on 64-bit Linux
print(np.array([1, 2.5]).dtype) # float64
df = pd.read_csv(io.StringIO('qty\n4\n7\n-\n'))
print(df['qty'].dtype) # object in pandas 2.x3. What do shape, ndim and size tell you, and how does reshape work?
shape is a tuple with the length of each dimension, ndim is the number of dimensions and size is the total number of elements. reshape gives the same data a new shape as long as the element count stays the same, and -1 asks NumPy to work out one dimension for you. For 12 values, reshape(3, 4) gives 3 rows of 4 and reshape(-1, 6) gives 2 rows of 6, but reshape(5, 3) raises a ValueError because 15 is not 12. reshape returns a view when it can, so writing to the result can change the original.
import numpy as np
a = np.arange(12) # 0 to 11
m = a.reshape(3, 4)
print(m.shape, m.ndim, m.size) # (3, 4) 2 12
print(a.reshape(-1, 6).shape) # (2, 6)4. What is broadcasting in NumPy?
Broadcasting is how NumPy combines arrays of different shapes without copying data. It compares shapes from the last dimension backwards, and two dimensions fit when they are equal or one of them is 1; a missing leading dimension counts as 1. A (3, 4) array of item prices times a (4,) array of tax rates works because the trailing 4s match, so each rate applies down its column. A (3, 4) array plus a (3,) array fails, because 4 and 3 differ. To add one value per row, reshape it to (3, 1).
import numpy as np
prices = np.full((3, 4), 100.0) # 3 orders, 4 items
rates = np.array([0.05, 0.12, 0.18, 0.28])
print((prices * rates).shape) # (3, 4)
extra = np.array([10, 20, 30]) # one value per order
print((prices + extra[:, None]).shape) # (3, 4)
# prices + extra raises ValueError: (3, 4) and (3,)5. What does vectorization mean in NumPy and pandas?
Vectorization means writing an operation on whole arrays or columns instead of a Python loop over the elements. The loop still happens, but inside compiled code that works on raw numbers, so NumPy does not create a Python object or run the interpreter for every value. Functions that work element by element this way are called ufuncs, such as np.sqrt, np.exp and the arithmetic operators. Adding 18 percent GST to a million amounts with amounts * 1.18 is far faster than a for loop or a list comprehension, and it is also shorter to read.
6. What is the difference between a Series and a DataFrame?
A Series is a one-dimensional labelled array: values of one dtype plus an index. A DataFrame is a two-dimensional table of columns that share one row index, and each column is a Series with its own dtype. Selecting one column with single brackets, df['city'], gives a Series, while double brackets, df[['city']], give a DataFrame with one column. The methods differ too: a Series has the .str and .dt accessors, while a DataFrame has merge and pivot_table. Many bugs come from passing a Series where a function expects a DataFrame.
import pandas as pd
df = pd.DataFrame({
'city': ['Pune', 'Indore', 'Kochi'],
'orders': [120, 85, 60],
})
print(type(df['city']).__name__) # Series
print(type(df[['city']]).__name__) # DataFrame
print(df.shape) # (3, 2)7. What happens when you add two Series with different indexes?
pandas aligns on index labels, not on position. It takes the union of both indexes, adds values whose labels match and puts NaN where a label exists in only one Series. Because NaN is a float, the integer result becomes float64. In the example only label b is in both, so it gets 21, and a and c get NaN. If missing labels should count as zero, use s1.add(s2, fill_value=0), which gives 10, 21 and 2. A common mistake is expecting position-wise addition; if that is really what you want, use .to_numpy() on both.
import pandas as pd
s1 = pd.Series([10, 20], index=['a', 'b'])
s2 = pd.Series([1, 2], index=['b', 'c'])
total = s1 + s2 # a NaN, b 21, c NaN
safe = s1.add(s2, fill_value=0) # a 10, b 21, c 28. What is the difference between loc and iloc?
loc selects by label: index labels for rows and names for columns. iloc selects by integer position, like a Python list. The key difference is slicing: a loc slice includes the end label, while an iloc slice excludes the end position, as in Python. With the default RangeIndex, df.loc[0:2] returns three rows and df.iloc[0:2] returns two. After filtering or sorting, labels and positions no longer match, so df.loc[0] can raise a KeyError while df.iloc[0] still returns the first row. Use loc with conditions and column names.
import pandas as pd
df = pd.DataFrame({'name': ['Asha', 'Ravi', 'Meena', 'Arjun'],
'score': [78, 91, 66, 84]})
print(len(df.loc[0:2])) # 3: labels 0, 1 and 2
print(len(df.iloc[0:2])) # 2: positions 0 and 1
print(df.loc[df['score'] > 80, 'name'].tolist())
# ['Ravi', 'Arjun']9. How do you filter rows on more than one condition in pandas?
Build a boolean mask for each condition and combine them with & for and, | for or and ~ for not, with each condition in brackets. The brackets are needed because & binds more tightly than == or >. Using the Python words and or or raises a ValueError saying the truth value of a Series is ambiguous, because Python cannot turn a whole column into one True or False. For a list of allowed values, use isin instead of a chain of ORs. The result keeps the original index labels, so call reset_index(drop=True) if you need 0, 1, 2 again.
import pandas as pd
df = pd.DataFrame({'city': ['Pune', 'Delhi', 'Pune', 'Surat'],
'amount': [450, 900, 1200, 700]})
big_pune = df[(df['city'] == 'Pune') & (df['amount'] > 500)]
west = df[df['city'].isin(['Pune', 'Surat'])]
print(len(big_pune), len(west)) # 1 310. You get a CSV of orders. What are the first things you do after loading it?
Load it with pd.read_csv, then check its size and structure before any analysis. shape gives rows and columns, head() shows a sample, info() lists each column's dtype and non-null count, and describe() gives quick statistics for numeric columns. Then look for problems: numbers read as object, dates read as text, missing values with isna().sum(), and duplicate IDs with duplicated().sum() on the key column. Catching a wrong dtype or a duplicate key at this stage saves you from wrong totals later, which is the mistake interviewers most want you to avoid.
11. How does pandas represent and handle missing values?
Float columns use NaN, and None becomes NaN in numeric data; datetime columns use NaT, and the nullable dtypes use pd.NA. Find them with isna() or notna(), never with == np.nan, because NaN is not equal to anything, not even itself. Reductions skip missing values by default, so sum and mean use the values that are there, and count() returns the number of non-missing values while len() counts all rows. Then decide what to do: dropna() removes rows and fillna() replaces values, and the right choice depends on why the data is missing.
import pandas as pd
s = pd.Series([10, None, 30])
print(s.isna().sum()) # 1
print(s.sum()) # 40.0
print(s.mean()) # 20.0
print(s.count(), len(s)) # 2 312. Why does an integer column become float after a missing value appears?
The classic NumPy int64 dtype has no way to store a missing value, and NaN is a floating-point value, so pandas stores the column as float64. IDs like 1001 then show as 1001.0, and integers above 2 to the power 53 can lose precision, because float64 cannot represent every integer beyond that. The fix is a nullable integer dtype: astype('Int64') with a capital I, or dtype='Int64' in read_csv, which keeps whole numbers and marks gaps as pd.NA. For codes with leading zeros, such as 007841, read the column as a string instead.
13. What does axis=0 and axis=1 mean in pandas and NumPy?
axis=0 works along the rows, down each column, and axis=1 works across the columns, along each row. So df.sum(axis=0), the default, gives one total per column, and df.sum(axis=1) gives one total per row. An easy way to remember it for reductions: the axis you name is the one that disappears. For drop, axis says which labels to remove: axis=0 drops rows and axis=1 drops columns, which is why df.drop(columns=['x']) is clearer. NumPy uses the same rule, so a (3, 4) array summed with axis=0 has shape (4,).
import pandas as pd
df = pd.DataFrame({'jan': [10, 20], 'feb': [30, 40]},
index=['Pune', 'Nagpur'])
print(df.sum(axis=0).tolist()) # [30, 70]: one per month
print(df.sum(axis=1).tolist()) # [40, 60]: one per city14. How do you sort data and count values in pandas?
sort_values sorts by one or more columns, with ascending set per column, for example sort_values(['city', 'amount'], ascending=[True, False]), and sort_index sorts by the index. For the top rows, nlargest(5, 'amount') is shorter than sorting and slicing. value_counts on a column counts each distinct value, from most to least frequent; normalize=True gives shares instead, and dropna=False also counts missing values, which are skipped by default. These methods return new objects, so assign the result, or the sort is lost.
15. Explain groupby in pandas with an example.
groupby follows split, apply and combine: it splits the rows into groups by one or more keys, applies a function to each group and combines the results. df.groupby('city')['amount'].sum() gives one total per city, with the city as the index. By default the group keys are sorted and rows with a missing key are left out; pass dropna=False to keep them as their own group. Use as_index=False or reset_index() when you want city as a normal column for the next step, such as a merge or an export to Excel.
import pandas as pd
df = pd.DataFrame({'city': ['Pune', 'Delhi', 'Pune', 'Delhi'],
'amount': [450, 900, 1200, 300]})
by_city = df.groupby('city')['amount'].sum()
print(by_city.index.tolist()) # ['Delhi', 'Pune']
print(by_city.tolist()) # [1200, 1650]16. What does pd.merge do, and what is the default join type?
merge combines two DataFrames on matching key values, like a SQL join. Name the key with on= when both tables use the same column name, or with left_on= and right_on= when the names differ. The default is how='inner', which keeps only keys found in both tables, so orders whose customer is missing from the customer table silently disappear. For a lookup that must keep every order, use how='left'. Non-key columns with the same name get the suffixes _x and _y unless you pass suffixes=. Compare row counts before and after.
import pandas as pd
orders = pd.DataFrame({'order_id': [1, 2, 3],
'cust_id': [101, 102, 104]})
custs = pd.DataFrame({'cust_id': [101, 102, 103],
'city': ['Pune', 'Delhi', 'Kochi']})
inner = orders.merge(custs, on='cust_id')
left = orders.merge(custs, on='cust_id', how='left')
print(len(inner), len(left)) # 2 317. When do you use concat instead of merge?
concat stacks objects. With axis=0, the default, it puts DataFrames one under another, such as monthly files with the same columns; with axis=1 it places them side by side, aligned on the index. merge, by contrast, joins rows on key values. With concat, a column that exists in only some frames is filled with NaN in the others, so check that the column names match first. Pass ignore_index=True to get a fresh 0 to n-1 index, or the labels from each file repeat. DataFrame.append was removed in pandas 2.0, so concat is the way to stack rows.
18. How do you work with dates in pandas?
Convert text to real datetimes with pd.to_datetime, or parse_dates in read_csv, so the column gets the datetime64 dtype. The .dt accessor then gives parts such as .dt.year, .dt.month and .dt.day_name(), and subtracting two dates gives a Timedelta, with .dt.days for whole days. Filters such as df[df['order_date'] >= '2026-04-01'] compare real dates, while on text they compare characters. Indian files often use DD/MM/YYYY, so pass format= or dayfirst=True; otherwise 03/04/2026 is read month first, as 4 March.
import pandas as pd
df = pd.DataFrame({'ordered': ['03/04/2026', '15/04/2026'],
'delivered': ['05/04/2026', '20/04/2026']})
for col in ['ordered', 'delivered']:
df[col] = pd.to_datetime(df[col], format='%d/%m/%Y')
days = (df['delivered'] - df['ordered']).dt.days
print(days.tolist()) # [2, 5]
print(df['ordered'].dt.month.tolist()) # [4, 4]19. What does apply do, and why is it often slower than a vectorized operation?
apply calls a Python function once per element, once per row with axis=1, or once per column. It is flexible, but every call runs in the Python interpreter, so on a large frame it can be many times slower than an equivalent column expression that runs in compiled code. df['qty'] * df['price'] multiplies every row in one step, while df.apply(lambda r: r['qty'] * r['price'], axis=1) also builds a Series object for every row. Use apply when no vectorized method exists, such as a call to a custom parser, and keep it out of hot paths.
20. In NumPy, what is the difference between a view and a copy?
A view is a new array object that looks at the same memory as the original, so writing to it changes the original. A copy owns its own memory. Basic slicing, such as a[1:4] or a[:, 0], returns a view, which is why slicing a large array is cheap. Fancy indexing with a list of positions and boolean masks return copies. When you need an independent array, call .copy() explicitly. A common bug is taking a slice, normalising it in place, and later finding that the original data changed too.
import numpy as np
a = np.array([10, 20, 30, 40, 50])
v = a[1:4] # basic slice: a view
v[0] = 99
print(a.tolist()) # [10, 99, 30, 40, 50]
c = a[[1, 2]] # fancy indexing: a copy
c[0] = 0
print(a.tolist()) # [10, 99, 30, 40, 50]Ready to check yourself? Take the free Pandas and NumPy mock test
Pandas and NumPy interview questions: Medium questions (1-3 years)
20 questions21. What does SettingWithCopyWarning mean, and how do you fix it?
pandas raises it when you assign into an object that may be a copy of another DataFrame, so the change may not reach the data you meant. The usual causes are chained indexing, like df[mask]['amount'] = 0, where the first step returns a copy, and adding a column to a filtered subset, like pune = df[mask] followed by a new pune['gst'] column. Fix the first with one loc call, df.loc[mask, 'amount'] = 0, and the second by calling .copy() when you create the subset. With copy-on-write turned on, this warning no longer appears.
import pandas as pd
df = pd.DataFrame({'city': ['Pune', 'Delhi', 'Pune'],
'amount': [450, 900, 1200]})
mask = df['city'] == 'Pune'
df[mask]['amount'] = 0 # chained: df is not changed
df.loc[mask, 'amount'] = 0 # one loc call: df changes
pune = df[mask].copy() # independent subset
pune['gst'] = pune['amount'] * 0.1822. How do you compute several aggregations per group with clear column names?
Use groupby with agg and named aggregation: each keyword becomes an output column, and its value is a pair of source column and function. This gives flat, readable names instead of a two-level column index that you must rename later. Built-in names such as 'sum', 'mean', 'count', 'nunique' and 'max' run in optimised code, so prefer them to lambdas. Pass as_index=False, or call reset_index(), to keep the group key as a column. Remember that 'count' skips missing values; use 'size' for the number of rows.
import pandas as pd
df = pd.DataFrame({
'city': ['Pune', 'Delhi', 'Pune', 'Delhi', 'Pune'],
'order_id': [1, 2, 3, 4, 5],
'cust_id': [11, 12, 11, 13, 14],
'amount': [450, 900, 1200, 300, 700],
})
summary = df.groupby('city', as_index=False).agg(
orders=('order_id', 'count'),
customers=('cust_id', 'nunique'),
revenue=('amount', 'sum'),
avg_order=('amount', 'mean'),
)
print(summary['revenue'].tolist()) # [1200, 2350]23. What is the difference between groupby agg and groupby transform?
agg reduces each group to one value, so the result has one row per group. transform returns a result with the same index and length as the original, with each group's value repeated on every row of that group. That makes transform the tool for putting a group-level number back on the detail rows, such as each order's share of its city's revenue, or filling gaps with the group median, without a merge. A common mistake is computing the agg result and then merging it back, which is longer and risks a wrong join.
import pandas as pd
df = pd.DataFrame({'city': ['Pune', 'Delhi', 'Pune', 'Delhi'],
'amount': [300, 900, 700, 100]})
g = df.groupby('city')['amount']
totals = g.agg('sum') # 2 rows, one per city
city_total = g.transform('sum') # 4 rows, like df
df['share'] = df['amount'] / city_total
print(df['share'].tolist()) # [0.3, 0.9, 0.7, 0.1]24. In a groupby, what is the difference between size and count?
size returns the number of rows in each group, including rows with missing values. count returns the number of non-missing values, per column for a DataFrame groupby. The two differ as soon as a column has gaps, and using the wrong one is a common reporting bug: counting orders with count on a coupon column counts only the orders that used a coupon. To keep or drop whole groups based on a property of the group, such as cities with at least 3 orders, use filter, which returns the original rows of the groups that pass.
import pandas as pd
df = pd.DataFrame({'city': ['Pune', 'Pune', 'Delhi', 'Pune'],
'coupon': ['NEW10', None, 'FEST', None]})
g = df.groupby('city')
print(g.size().tolist()) # [1, 3]: Delhi, Pune
print(g['coupon'].count().tolist()) # [1, 1]
busy = g.filter(lambda x: len(x) >= 3)
print(len(busy)) # 325. Explain the how= options of merge and how to check what matched.
inner keeps keys present in both tables, left keeps every row of the left table and fills unmatched right columns with NaN, right does the opposite, and outer keeps all keys from both sides; cross pairs every left row with every right row. Pass indicator=True to get a _merge column that says left_only, right_only or both. It is the quickest way to find ledger entries missing from the bank file, or the reverse. Use left for lookups where the left table must not lose rows, and outer when reconciling two sources.
import pandas as pd
ledger = pd.DataFrame({'txn': ['T1', 'T2', 'T3'],
'amount': [500, 750, 120]})
bank = pd.DataFrame({'txn': ['T2', 'T3', 'T4'],
'credited': [750, 120, 999]})
rec = ledger.merge(bank, on='txn', how='outer',
indicator=True)
print(len(rec)) # 4
print(rec['_merge'].value_counts()['both']) # 226. After a left merge your table has more rows than before. Why, and how do you prevent it?
A left merge keeps every left row, but if a key appears more than once in the right table, the left row is repeated once per match. Duplicate keys on the lookup side, such as two rows for the same customer after a bad data load, multiply your orders and inflate every total computed afterwards. Check the key first with custs['cust_id'].duplicated().sum(), deduplicate if needed, and pass validate='many_to_one' so pandas raises a MergeError instead of duplicating silently. Then compare len() before and after.
import pandas as pd
orders = pd.DataFrame({'order_id': [1, 2, 3],
'cust_id': [101, 101, 102]})
custs = pd.DataFrame({'cust_id': [101, 101, 102],
'city': ['Pune', 'Pune', 'Delhi']})
out = orders.merge(custs, on='cust_id', how='left')
print(len(out)) # 5: orders 1 and 2 match twice each
orders.merge(custs, on='cust_id', how='left',
validate='many_to_one') # raises MergeError27. How is DataFrame.join different from merge, and when do you use concat with axis=1?
join is a shortcut that matches on the index of the other frame, and on the caller's index unless you pass on=, with how='left' as its default rather than inner. merge matches on columns by default and can use indexes through left_index and right_index. concat with axis=1 lines up several Series or frames side by side by index alignment alone, which suits monthly totals that share a city index. join and concat align on index labels, so keys of different types, such as 101 and '101', never match and give NaN, while merge on an int and a string key column raises a ValueError.
28. What is the difference between pivot and pivot_table?
Both turn long data into a wide table with one row per index value and one column per value of another column. pivot only reshapes, so it raises a ValueError if an index and column pair appears more than once. pivot_table aggregates duplicates with aggfunc, which defaults to 'mean', so pass aggfunc='sum' explicitly for revenue. It also offers fill_value for empty cells and margins=True for row and column totals. Use pivot when each pair is unique by design, and pivot_table for report-style summaries such as sales by city and month.
import pandas as pd
df = pd.DataFrame({
'city': ['Pune', 'Pune', 'Delhi', 'Pune'],
'month': ['Jan', 'Jan', 'Jan', 'Feb'],
'amount': [450, 300, 900, 700],
})
report = df.pivot_table(index='city', columns='month',
values='amount', aggfunc='sum',
fill_value=0)
print(report.shape) # (2, 2)
# df.pivot(index='city', columns='month', values='amount')
# raises ValueError: Pune and Jan appear twice29. How and why do you use melt?
melt turns a wide table into a long one. The identifier columns stay, and each remaining column is turned into rows that hold the old column name in one column and its value in another. Wide data is common in Excel exports, with one column per month, but groupby, plotting and resampling work best on long data with one value per row. id_vars names the columns to keep, value_vars the columns to unpivot, and var_name and value_name name the new columns. A frame with 3 rows and 3 month columns becomes 9 rows; pivot reverses it.
import pandas as pd
wide = pd.DataFrame({'branch': ['Pune', 'Nashik', 'Thane'],
'jan': [450, 300, 520],
'feb': [700, 410, 610],
'mar': [650, 380, 590]})
long = wide.melt(id_vars='branch', var_name='month',
value_name='sales')
print(long.shape) # (9, 3)30. Dates in your CSV are in DD-MM-YYYY format and some are invalid. How do you parse them reliably?
Pass an explicit format, such as format='%d-%m-%Y', to pd.to_datetime. It is faster and removes any doubt between 3 April and 4 March. Add errors='coerce' so invalid entries like 31-02-2026 or the word pending become NaT instead of raising an error, then count and inspect them with isna(). Since pandas 2.0, to_datetime infers one format from the first valid value and applies it to the whole column, so a column with mixed formats needs format='mixed' or a cleaning step first. Report how many rows were coerced; never drop them silently.
31. How do you turn daily sales into monthly totals with resample?
resample groups rows into regular time buckets and needs a DatetimeIndex, or a datetime column passed with on=. df.resample('ME', on='date')['amount'].sum() gives one total per calendar month, labelled with the month-end date, and a month with no sales still appears with a sum of 0. In pandas 2.2 the month-end alias is 'ME'; the old 'M' still works but is deprecated, and 'MS' labels each month by its first day. For weeks use 'W', which ends on Sunday by default. resample works like a groupby for time, so agg, mean and count all work.
32. How do you replace a row-wise apply that contains if-else logic with a vectorized version?
Write each condition as a boolean mask over whole columns, then pick values with np.where for two outcomes or np.select for several. np.select checks the conditions in order and uses the first one that is true, with default for the rest, exactly like an if-elif-else chain. Because no Python function is called per row, it is usually many times faster than apply with axis=1 on a few lakh rows. Keep the conditions in a named list next to their values, so a business rule such as a delivery fee slab stays readable and easy to change.
import numpy as np
import pandas as pd
df = pd.DataFrame({'amount': [150, 450, 1200, 799]})
conds = [df['amount'] < 199,
df['amount'] < 799]
fees = [40, 20]
df['fee'] = np.select(conds, fees, default=0)
print(df['fee'].tolist()) # [40, 20, 0, 0]33. What is the difference between Series.map, Series.replace and Series.apply?
map transforms every value with a dict, a Series or a function; with a dict, values that are not keys become NaN. replace swaps only the values it finds and leaves everything else unchanged, which suits fixing a few spellings. apply calls a function per element and is the most flexible and the slowest of the three. So mapping state codes such as MH and KA to names with map turns an unknown code into NaN, a useful signal of a gap in the lookup table, while replace would keep the raw code. Use map when every value must be translated.
34. An amount column holds values like 'Rs 1,200', ' 850 ' and '-'. How do you clean it?
Use the .str accessor to remove the currency text, commas and spaces, then convert with pd.to_numeric(errors='coerce') so placeholders like - become NaN instead of stopping the job. Count the coerced values and look at them before you decide to fill or drop them. Clean text columns the same way: strip spaces and fix case so 'pune ' and 'Pune' fall into one group. Note that .str methods return NaN for missing values in object columns, so pass na=False to str.contains when you filter, or the mask cannot be used.
import pandas as pd
amt = pd.Series(['Rs 1,200', ' 850 ', '-', 'Rs 99'])
clean = (amt.str.replace('Rs', '', regex=False)
.str.replace(',', '', regex=False)
.str.strip())
num = pd.to_numeric(clean, errors='coerce')
print(num.tolist()) # [1200.0, 850.0, nan, 99.0]35. How do you find and remove duplicate records correctly?
First decide what a duplicate means: the same full row, or the same business key such as order_id. duplicated(subset=['order_id']) flags repeats, and keep='first', 'last' or False controls which copies count as repeats; keep=False marks every copy, which helps when you review them. To keep the latest version of each order, sort by the update time and call drop_duplicates(subset='order_id', keep='last'). Dropping only identical full rows misses records where just a timestamp differs, a common reason totals still look inflated.
36. When does the category dtype help, and when can it hurt?
category stores each distinct value once and keeps a small integer code per row, so a column with few distinct values repeated many times, like city or state in a million orders, can shrink many times over and group faster. It hurts when nearly every value is unique, such as order IDs, because you pay for both the codes and the categories. Watch two behaviours: in pandas 2.x groupby on a categorical defaults to observed=False, so unused categories appear as empty groups, and assigning a value that is not a category raises an error.
37. A 3 GB CSV does not fit comfortably in memory. How do you load it?
Read only what you need. usecols drops unused columns while parsing, dtype sets small types up front, such as int32 or category, and parse_dates converts dates in the same pass. If the data still does not fit, use chunksize to read and aggregate it in pieces, keeping only small partial results. engine='pyarrow' can parse faster on multi-core machines. Try the settings with nrows=100000 first. For files you reload often, convert the CSV once to Parquet, which stores the types and loads much faster than parsing text again.
38. How can you tell whether a NumPy operation returned a view or a copy?
Use np.shares_memory(a, b), or check b.base, which points to the owner of the memory for a view and is None for an array that owns its data. Basic slices, transposes, and reshape or ravel of a contiguous array give views; fancy indexing, boolean masks, flatten and most arithmetic give copies. Note the asymmetry: a[mask] on the right side of an assignment makes a copy, but a[mask] = 0 on the left side writes into a itself. That is why a[mask] *= 2 changes a, while b = a[mask] followed by b *= 2 does not.
import numpy as np
a = np.arange(6)
s = a[2:5]
f = a[[2, 3, 4]]
print(np.shares_memory(a, s)) # True: a slice is a view
print(np.shares_memory(a, f)) # False: fancy index copies
a[a > 3] = 0 # writes into a
print(a.tolist()) # [0, 1, 2, 3, 0, 0]39. How do you subtract each row's mean from a 2-D array, and why does keepdims help?
a.mean(axis=1) returns one mean per row with shape (n,), and broadcasting lines that up against the last axis, the columns, not the rows. For a square array this silently gives a wrong answer, and for other shapes it raises an error. keepdims=True keeps the reduced axis with size 1, shape (n, 1), so the means broadcast across each row correctly. The same applies to dividing by row totals to get shares. Work along axis=0 broadcasts correctly even without keepdims, because the reduced shape matches the trailing axis.
import numpy as np
marks = np.array([[60, 70, 80],
[50, 50, 80]]) # 2 students, 3 tests
row_mean = marks.mean(axis=1, keepdims=True)
print(row_mean.shape) # (2, 1)
centred = marks - row_mean
print(centred.tolist())
# [[-10.0, 0.0, 10.0], [-10.0, -10.0, 20.0]]40. How do you fill missing values sensibly instead of using one global value?
Choose the fill from the meaning of the data. For a measure that varies by group, fill with that group's median using groupby and transform, so a missing delivery time in Kochi gets Kochi's typical value, not the national one. For time series such as daily stock levels, ffill carries the last known value forward, and interpolate fills gaps in smooth readings. Text gaps often deserve an explicit Unknown label. Whatever you choose, record which values were missing first, in a flag column, so later analysis can tell real values from filled ones.
import pandas as pd
df = pd.DataFrame({
'city': ['Kochi', 'Kochi', 'Kochi', 'Pune', 'Pune'],
'days': [2.0, None, 4.0, 5.0, None],
})
df['days_missing'] = df['days'].isna()
median = df.groupby('city')['days'].transform('median')
df['days'] = df['days'].fillna(median)
print(df['days'].tolist()) # [2.0, 3.0, 4.0, 5.0, 5.0]Ready to check yourself? Take the free Pandas and NumPy mock test
Pandas and NumPy interview questions: Hard questions (experienced)
20 questions41. What is copy-on-write in pandas 2.x, and what changes when you enable it?
With copy-on-write, every DataFrame or Series derived from another one behaves as an independent copy, but pandas shares the memory lazily and copies only when one side is modified. So a column you pulled out, a subset or a method result can no longer change its parent, and changing the parent no longer changes them. Chained assignment never updates the original. In pandas 2.x you turn it on with pd.options.mode.copy_on_write = True, and pandas 2.2 adds a 'warn' setting to find code whose behaviour would change. pandas 3.0 makes it the default.
import pandas as pd
pd.options.mode.copy_on_write = True
orders = pd.DataFrame({'qty': [2, 5, 1],
'price': [250, 120, 999]})
qty = orders['qty'] # shares memory, no copy yet
qty.iloc[0] = 0 # the copy happens here
print(orders['qty'].tolist()) # [2, 5, 1]
orders.loc[0, 'qty'] = 0 # the way to change orders42. How does NumPy store an array in memory, and what are strides?
An ndarray is a header over a flat block of bytes: a dtype, a shape, and strides, the number of bytes to step to reach the next element along each axis. A C-ordered (3, 4) int64 array has strides (32, 8): 8 bytes to the next column and 32 to the next row. Transposing just swaps the strides, so a.T is a view created instantly, and many slices only change the strides and the start offset. The catch is that non-contiguous views can be slower to process, and reshaping one may need a copy; np.ascontiguousarray makes a compact copy.
import numpy as np
a = np.zeros((3, 4), dtype=np.int64)
print(a.strides) # (32, 8)
print(a.T.strides) # (8, 32)
print(a[:, ::2].strides) # (32, 16)
print(a.flags['C_CONTIGUOUS'], a.T.flags['C_CONTIGUOUS'])
# True False43. A colleague's code subtracts two arrays and suddenly needs gigabytes of memory. What could be wrong?
The usual cause is an accidental broadcast between shapes (n,) and (n, 1), which gives an (n, n) result instead of n values. It often comes from model predictions shaped as a column, or a column taken with a[:, [0]], compared against a flat array of targets. With 100,000 rows the result has 10 billion elements, about 80 GB in float64. Check shapes before element-wise work, for example with an assert, and flatten with ravel() or take columns with a[:, 0] to stay 1-D. An error from mismatched shapes is far safer than a silent broadcast.
import numpy as np
y_true = np.zeros(5) # shape (5,)
y_pred = np.zeros((5, 1)) # shape (5, 1)
err = y_true - y_pred
print(err.shape) # (5, 5), not (5,)
err = y_true - y_pred.ravel()
print(err.shape) # (5,)44. Can NumPy integer arithmetic overflow, and what changed about type promotion in NumPy 2?
Yes. Element-wise arithmetic on fixed-size integer arrays wraps around silently, unlike Python ints, which grow without limit. An int8 array holding 100 doubled gives -56, and an int32 product of a price in paise and a quantity can pass 2,147,483,647 and wrap. Choose int64 early for such values. NumPy 2 also adopted NEP 50: Python scalars no longer change the result type based on their value, so np.float32(3) + 3.0 stays float32 instead of becoming float64, and the default integer on 64-bit Windows is now int64. Old code can change dtype after an upgrade.
import numpy as np
a = np.array([100, 120], dtype=np.int8)
print((a * 2).tolist()) # [-56, -16], no error
paise = np.array([50_000_000], dtype=np.int32)
qty = np.array([50], dtype=np.int32)
print((paise * qty).dtype) # int32: the product wraps
x = np.float32(3) + 3.0
print(type(x).__name__) # float32 in NumPy 245. How do you measure and cut the memory used by a large DataFrame?
Start with df.info(memory_usage='deep') or df.memory_usage(deep=True), because without deep=True object columns are counted as 8-byte pointers and the strings themselves are ignored. Then fix the largest columns: downcast numbers with pd.to_numeric(downcast='integer') where the range allows, convert low-cardinality text to category, and consider Arrow-backed types with dtype_backend='pyarrow' in read_csv, available from pandas 2.0. Drop unneeded columns before merges and concat, because those build new frames and briefly hold the old and new data together.
46. You process a huge CSV in chunks to get the average order value per city. What is the trap?
Averaging the per-chunk averages is wrong whenever chunks hold different numbers of rows for a city. Keep the sum and the count per chunk, add them up across chunks and divide once at the end. For example, if Pune has two orders of 100 in one chunk and one of 400 in another, the mean of the chunk means is 250 but the true average is 200. Medians and distinct counts cannot be built from partial numbers like these at all; they need all the values or the sets of distinct values, an approximate method, or a tool such as DuckDB.
import pandas as pd
parts = []
for chunk in pd.read_csv('orders.csv', chunksize=500_000,
usecols=['city', 'amount']):
parts.append(chunk.groupby('city')['amount']
.agg(['sum', 'count']))
total = pd.concat(parts).groupby(level=0).sum()
avg = total['sum'] / total['count']47. Why is groupby.apply often slow, and what do you use instead?
apply calls a Python function once per group and then has to work out how to stitch the results together, so with thousands of groups the overhead dominates. Built-in reductions and transforms such as sum, mean, cumsum, rank and transform('max') run in compiled code over all groups at once. Rewrite the logic in those terms: compute group statistics with transform, then use plain column arithmetic. In pandas 2.2, apply passing the grouping columns to your function is deprecated; pass include_groups=False. Keep apply for logic that truly needs a whole group.
import pandas as pd
df = pd.DataFrame({'cust': ['A', 'A', 'B', 'B', 'B'],
'amount': [100, 300, 50, 50, 200]})
# slow: one Python call per customer
slow = df.groupby('cust')[['amount']].apply(
lambda g: g['amount'] > g['amount'].mean())
# fast: group mean broadcast back, then compare
avg = df.groupby('cust')['amount'].transform('mean')
df['above_avg'] = df['amount'] > avg
print(df['above_avg'].tolist())
# [False, True, False, False, True]48. How do you join each trade to the most recent price at or before its time?
Use pd.merge_asof, a left join on the nearest key rather than an exact one. With the default direction='backward', it takes the last right row whose key is less than or equal to the left key, which is exactly the latest price known at trade time, and it never looks into the future. Both frames must be sorted by the on key, by= restricts matches to the same instrument, and tolerance= stops a stale price from long ago being used. A plain merge on timestamps would match almost nothing, because exact times rarely line up.
import pandas as pd
day = '2026-04-06 '
trades = pd.DataFrame({
'time': pd.to_datetime([day + '09:15:02', day + '09:15:07',
day + '09:16:30']),
'qty': [10, 5, 8],
})
quotes = pd.DataFrame({
'time': pd.to_datetime([day + '09:15:00', day + '09:15:05',
day + '09:16:00']),
'price': [101.5, 101.8, 102.1],
})
out = pd.merge_asof(trades, quotes, on='time',
tolerance=pd.Timedelta('1min'))
print(out['price'].tolist()) # [101.5, 101.8, 102.1]49. How do you handle time zones in pandas, for example IST times from an app and UTC times from a server log?
Make every timestamp time zone aware, then compare in one zone. tz_localize attaches a zone to naive times without changing the clock reading, and tz_convert moves aware times to another zone while keeping the same instant. So app times recorded in India get tz_localize('Asia/Kolkata'), server logs get tz_localize('UTC'), and both are converted to one zone before a join. Comparing naive and aware values raises a TypeError, which is a useful guard. India has no daylight saving, but many partner countries do, so never add a fixed offset by hand.
import pandas as pd
app = pd.to_datetime(pd.Series(['2026-03-01 09:30']))
app = app.dt.tz_localize('Asia/Kolkata')
utc = app.dt.tz_convert('UTC')
print(utc.dt.hour.tolist(), utc.dt.minute.tolist())
# [4] [0]50. When do you use resample, groupby with pd.Grouper, and rolling on time series data?
resample buckets one time series into fixed periods, such as weekly totals. groupby with pd.Grouper(key='date', freq='ME'), using the pandas 2.2 alias, does the same bucketing combined with other keys, such as month and city in one pass. rolling gives a moving window that slides row by row, such as a 7-day moving average of daily orders. With a DatetimeIndex, rolling('7D') uses a time window, so missing days do not stretch it the way rolling(7) over rows would. min_periods sets how many values a window needs before it returns a number instead of NaN.
51. Are pandas string methods vectorized in the same sense as numeric operations?
Not fully. With the default object dtype in pandas 2.x, each .str method still loops over Python string objects one by one. It is convenient and handles missing values, but it is far slower than numeric work. Arrow-backed strings, dtype='string[pyarrow]', run many string operations in compiled Arrow code and use less memory. Other speed-ups: filter first so fewer strings are processed, clean only the distinct values once and map the results back, which helps when a few city names repeat a million times, and use one regex with str.extract instead of several passes.
52. Some logic depends on the previous row's result, like a wallet balance that cannot go below zero. How do you handle it efficiently?
First check whether a built-in cumulative function fits: cumsum, cumprod, cummax, shift and diff cover running totals, streaks and changes without a loop. A rule like a balance that never goes below zero makes each step depend on the clipped previous result, so one cumsum cannot express it. Then a loop is acceptable, but run it over plain NumPy arrays from to_numpy(), not iterrows, which builds a Series for every row. If it is still too slow on millions of rows, compile the same loop with Numba. Write the result back as one new column.
import numpy as np
import pandas as pd
df = pd.DataFrame({'change': [500, -200, -400, 300, -100]})
ch = df['change'].to_numpy()
bal = np.empty(len(ch))
cur = 0
for i in range(len(ch)):
cur = max(cur + ch[i], 0)
bal[i] = cur
df['balance'] = bal
print(bal.tolist()) # [500.0, 300.0, 0.0, 300.0, 200.0]53. What is a MultiIndex, and how do stack and unstack relate to it?
A MultiIndex is an index with several levels, which you get from a groupby on two keys such as city and month. Select from it with loc and a tuple, such as s.loc[('Pune', 'Feb')], or with xs for a single level. unstack moves an index level into the columns, turning a long city-month Series into a city by month table, and stack does the reverse. Combinations that do not exist become NaN after unstack, so pass fill_value=0 for counts. Many teams call reset_index() early, because flat columns are easier to merge, export and explain.
import pandas as pd
df = pd.DataFrame({
'city': ['Pune', 'Pune', 'Delhi'],
'month': ['Jan', 'Feb', 'Jan'],
'orders': [12, 9, 20],
})
s = df.groupby(['city', 'month'])['orders'].sum()
print(s.loc[('Pune', 'Feb')]) # 9
table = s.unstack(fill_value=0)
print(table.shape) # (2, 2)54. How do you structure a pandas cleaning pipeline that a team can trust and rerun?
Write it as small pure functions that take a DataFrame and return a new one, and chain them with pipe and assign, so each step is named, testable and easy to reorder. Validate at the boundaries: assert the expected columns and dtypes after loading, check key uniqueness before merges and row counts after them. Test each step on a tiny hand-made DataFrame with pandas.testing.assert_frame_equal. Avoid inplace=True and global state, log how many rows each step drops, and keep the raw files read-only so a rerun gives the same output.
55. Walk through a typical data-cleaning task: a sales export from 20 branches with messy city names, dates and amounts.
Profile first: shape, dtypes, isna().sum() and value_counts on the key text columns. Read everything as text, then fix types: strip and standardise city names with a mapping for known spellings, parse dates with an explicit day-first format and errors='coerce', and turn amounts into numbers with to_numeric. Then remove duplicate order_id rows, keeping the latest, and decide per column how to treat gaps. Check rules such as no negative amounts, and set failing rows aside instead of deleting them. Finish with counts of rows in, fixed and rejected.
import pandas as pd
df = pd.read_csv('branch_sales.csv', dtype=str)
fix = {'bombay': 'Mumbai', 'bangalore': 'Bengaluru',
'gurgaon': 'Gurugram'}
city = df['city'].str.strip().str.lower()
df['city'] = city.map(fix).fillna(city.str.title())
df['date'] = pd.to_datetime(df['date'], format='%d/%m/%Y',
errors='coerce')
df['amount'] = pd.to_numeric(
df['amount'].str.replace(',', '', regex=False),
errors='coerce')
bad = df['date'].isna() | df['amount'].isna() | (df['amount'] < 0)
rejected, clean = df[bad], df[~bad]
clean = clean.drop_duplicates(subset='order_id', keep='last')56. Is inplace=True faster, and why might a function change the caller's DataFrame?
Usually it is not faster: most methods build a new result internally and then swap it into the object, so inplace=True saves little time or memory, while it blocks method chaining and returns None, which causes bugs like df = df.dropna(inplace=True). The other surprise is that a DataFrame passed into a function is the same object, so df['x'] = ... inside the function changes the caller's data. Copy-on-write does not prevent this, because it is a change to the object itself, not to a derived one. Use assign, or start with df = df.copy().
import pandas as pd
def add_gst(df):
df['gst'] = df['amount'] * 0.18 # changes the caller's df
return df
def add_gst_safe(df):
return df.assign(gst=df['amount'] * 0.18) # new frame
orders = pd.DataFrame({'amount': [100, 250]})
add_gst_safe(orders)
print(list(orders.columns)) # ['amount']
add_gst(orders)
print(list(orders.columns)) # ['amount', 'gst']57. How does pd.NA behave in comparisons and logic, and how is it different from NaN?
pd.NA is the missing marker of the nullable dtypes such as Int64, boolean and string. NaN is a float, while NA works across those dtypes and follows three-valued Kleene logic: a comparison with NA gives NA, NA or True is True, NA and False is False, and NA and True stays NA, because the result is known only when the missing value cannot change it. Calling bool() on NA raises a TypeError, so if x == pd.NA fails loudly. In boolean indexing, pandas treats NA in a nullable mask as False, so those rows are dropped.
import pandas as pd
paid = pd.Series([True, pd.NA, False], dtype='boolean')
vip = pd.Series([True, True, pd.NA], dtype='boolean')
either = paid | vip # True, True, NA
both = paid & vip # True, NA, False
s = pd.Series([10, 20, 30])
print(s[both].tolist()) # [10]: NA counts as False58. A daily report script has grown from 2 minutes to 40. How do you find and fix the slowness?
Measure before changing anything: time each step, or run cProfile or line_profiler, to find the one or two steps that dominate. Typical culprits are row loops or apply with axis=1, concat inside a loop, which copies everything collected so far on every pass and grows quadratically, repeated reads of the same file, and object columns that should be numbers or categories. Fix the biggest one, measure again and stop when it is fast enough. Filtering rows and dropping columns early also shrinks every later step, including merges and groupbys.
59. How do in-place operations and out= help NumPy performance, and what casting error can they raise?
An expression like a * 2 + 1 may allocate intermediate arrays, which costs memory and bandwidth on large data; NumPy avoids some of them automatically, but augmented assignment such as a *= 2, or a ufunc call with out=a, writes into existing memory explicitly. The catch is casting: an in-place operation cannot change the dtype, so an int64 array multiplied by 1.5 in place raises a UFuncTypeError under the default same_kind rule, while a = a * 1.5 simply creates a new float64 array. In-place changes are also visible through every view of that array.
60. When would you move from CSV and pandas to Parquet, Polars, DuckDB or Spark?
Change the file format first: Parquet is columnar, compressed and typed, so reads can skip unused columns and no type guessing happens, which usually cuts load time and disk use a lot compared with CSV. Stay with pandas while the data fits in memory with room to spare, since a pandas workflow often needs several times the data size in RAM. Beyond that, DuckDB runs SQL over files larger than memory on one machine, Polars offers a fast multi-threaded DataFrame API with lazy queries, and Spark suits data spread over a cluster.
Practice questions for November 2026
New every month3 questions from the Pandas and NumPy mock test, the way they appear in it. A new set comes on the 1st of every month. Pick your answer, then open it.
-
How many rows does df.head() return for a DataFrame with 100 rows?
- 3
- 10
- 5
- 1
Show the answer
Answer: 5. head() returns the first n rows, and n defaults to 5, so you get rows 0 to 4. Pass a number, as in df.head(10), for more. tail() works the same way from the end; both are quick checks that the data loaded as expected.
-
What is res.shape after this code runs?
import pandas as pd df = pd.DataFrame({ 'name': ['Asha', 'Ravi', 'Om', 'Iqbal'], 'city': ['Pune', 'Goa', 'Pune', 'Agra'], 'pay': [50, 60, 55, 40], }) res = df.loc[df['city'] == 'Pune', ['name', 'pay']]- (2, 2)
- (2, 3)
- (2,)
- (4, 2)
Show the answer
Answer: (2, 2). The mask keeps the 2 Pune rows, and the column list keeps name and pay. A list of columns always returns a DataFrame, so res.shape is (2, 2). Passing a single column label instead would return a Series of shape (2,).
-
What is s after this code runs?
import numpy as np a = np.zeros((3, 4), dtype=np.int64) s = a.strides- (4, 1)
- (8, 32)
- (32, 8)
- (24, 8)
Show the answer
Answer: (32, 8). Strides are the bytes to step to the next element along each axis. In C order the last axis is contiguous, so one step along it is one int64, 8 bytes, and a step to the next row skips 4 values, 32 bytes. Hence (32, 8).
Ready to apply?
Browse the open jobs on CoreHRx. Create a free profile once and apply with it.