An algorithm is a step-by-step method for turning an input into a required output. Studying algorithms means learning to compare possible methods, choose structures that support them, and explain why their steps produce the right result.
The difficulty for a beginner is often the order rather than any single topic. A definition may be simple but still feel ungrounded when the concrete problem it explains has not appeared yet. An advanced technique may look arbitrary when its supporting data structure or design idea is still missing.
The numbered stages below are the study order of this article. Follow the stages in order, and follow the topics within each stage in order. Each stage introduces the concepts needed by later stages, so the full map is the learning path rather than a separate catalog of topics.
The path assumes that the reader can follow variables, branches, loops, functions, and basic Python containers. Readers transferring from C, Java, or JavaScript can use Algorithm-Focused Python Syntax for C, Java, or JavaScript Programmers as a translation reference.
1. Run a Complete Algorithm
Begin with data that can be inspected directly and an algorithm that can be traced from start to finish. This gives later analysis terms something concrete to describe.
1.1 Array
An array stores elements in a sequence and provides access by index. Learn it first because searching, sorting, prefix sums, two pointers, and many dynamic programming states all operate on indexed sequences.
1.2 Linear Search
Linear search checks elements one by one until it finds the target or reaches the end. It is the first complete algorithm because its input, steps, result, and cost are all easy to see, and it becomes the baseline that later techniques improve.
2. Analyze Cost and Correctness
Once a complete scan is visible, learn how to describe what grows, how much the algorithm costs, and why its result can be trusted.
2.1 Input Size
Input size is the quantity that controls how much work an algorithm may perform. Identify it before using complexity notation because n may mean array length in one problem and the numbers of vertices and edges in another.
2.2 Big-O Notation
Big-O notation expresses an asymptotic upper bound on growth while ignoring constants and lower-order terms. It provides a shared vocabulary for distinguishing patterns such as logarithmic, linear, and quadratic growth.
2.3 Time Complexity
Time complexity describes how the amount of work grows with the input size. Use linear search as the first example, then compare later algorithms by their growth rather than by timing them on one computer.
2.4 Space Complexity
Space complexity describes how much additional memory an algorithm needs as the input grows. Study it after time complexity so algorithm choice becomes a comparison of both major resources instead of speed alone.
2.5 Correctness
Correctness means that an algorithm produces the intended result for every valid input covered by its contract. Complexity only compares usable solutions, so correctness must be established before efficiency can decide between them.
2.6 Loop Invariant
A loop invariant is a condition that remains true before and after each iteration. It turns correctness into a step-by-step argument and prepares the reader to justify insertion sort and binary search rather than only memorize their code.
3. Order Data and Find Values Faster
Linear search establishes the cost of checking everything. The next topics show how ordering or direct lookup can avoid repeated scanning.
3.1 Sorting
Sorting rearranges values into an order that later operations can exploit. Learn the purpose of sorting before individual sorting algorithms so the cost of creating order can be weighed against the work that order saves.
3.2 Insertion Sort
Insertion sort grows a sorted prefix one element at a time. It is the first sorting implementation because every step is visible and its sorted-prefix invariant provides a concrete correctness argument.
3.3 Binary Search
Binary search repeatedly discards half of a sorted search interval. It comes after sorting and loop invariants because its speed depends on order and its correctness depends on preserving the interval that can still contain the target.
3.4 Counting Sort
Counting sort records how often each value occurs instead of comparing pairs of values. It shows that sorting can beat the comparison model when the value range is small enough, while also making the constraint on that range explicit.
3.5 Hash Map
A hash map associates keys with values and usually supports lookup, insertion, and update in expected constant time. It gives a different way to avoid repeated scans when a problem needs counts, groups, indexes, or cached results rather than sorted order.
3.6 Hash Set
A hash set stores unique values and usually supports expected constant-time membership checks. Learn it after hash maps because it uses the same lookup idea when only presence or prior visitation matters.
4. Learn the Core Supporting Data Structures
Later algorithms require more than indexed storage and key lookup. Learn these structures now so recursion, traversal, priority-based selection, and hierarchical problems can be understood in terms of their actual operations.
4.1 Linked List
A linked list stores a sequence as nodes connected by references. It contrasts indexed arrays with link-based structure and makes explicit what is gained or lost when local insertion matters more than direct indexing.
4.2 Stack
A stack processes the most recently added item first. Its last-in, first-out behavior prepares the reader for function calls, recursion, DFS, backtracking, and monotonic stacks.
4.3 Queue
A queue processes the earliest added item first. Its first-in, first-out behavior is the mechanism behind layer-by-layer BFS and later supports monotonic queues.
4.4 Heap
A heap maintains partial order so the smallest or largest element can be accessed efficiently. Learn the representation before the priority-queue abstraction that uses it.
4.5 Priority Queue
A priority queue removes the currently highest- or lowest-priority item rather than the oldest item. It prepares the reader for greedy scheduling, Dijkstra’s algorithm, and Prim’s algorithm.
4.6 Tree
A tree represents hierarchical relationships without cycles. It connects links, recursion, and traversal, and it becomes a prerequisite for tries, tree dynamic programming, and the distinction between a tree and a general graph.
5. Reuse Work on Sequences
With arrays, lookup structures, stacks, and queues available, learn patterns that keep useful information while a sequence is scanned. These techniques replace repeated computation with maintained state.
5.1 Prefix Sum
A prefix sum stores cumulative totals so a range sum can be obtained from two stored values. It is the first preprocessing technique because it clearly trades an initial linear pass and extra memory for faster repeated queries.
5.2 Difference Array
A difference array stores changes between neighboring positions so many range updates can be recorded cheaply and accumulated later. It follows prefix sums because the two techniques use cumulative information in opposite directions.
5.3 Two Pointers
Two pointers coordinate two positions while scanning a sequence or interval. After arrays and sorting are familiar, the technique shows how a controlled movement rule can replace a nested search.
5.4 Sliding Window
A sliding window maintains information about a moving contiguous range. It follows two pointers because its left and right boundaries are pointers with the additional rule that the current range’s state is updated instead of recomputed.
5.5 Monotonic Stack
A monotonic stack keeps unresolved candidates in increasing or decreasing order. It extends the basic stack by discarding candidates that a newer value makes permanently useless, which solves nearest-greater, nearest-smaller, and span-like problems in linear time.
5.6 Monotonic Queue
A monotonic queue keeps the best remaining candidates in value order while also expiring positions that leave a window. Study it after sliding windows and monotonic stacks because it combines both ideas for moving-window minima or maxima.
6. Use Recursion to Explore Search Spaces
The next stage moves from one-direction scans to problems defined by smaller subproblems or multiple choices. The key is to make every possible computation state explicit before trying to optimize it.
6.1 Recursion
Recursion solves a problem by calling the same procedure on a smaller instance and stopping at a base case. It comes after stacks and trees because the call stack and hierarchical problem shape make the mechanism easier to trace.
6.2 Brute Force
Brute force tries every candidate allowed by the problem. It is the correct baseline for a search problem because it defines what must be considered before pruning, reuse, or a greedy choice can reduce the work.
6.3 Search Space
A search space is the set of candidates or computation states an algorithm may explore. Naming it makes complexity visible and provides a common model for brute force, backtracking, graph traversal, and dynamic programming.
6.4 State
A state contains exactly the information needed to describe one point in a computation and determine the valid next steps. It must be clear before search paths can be pruned or repeated subproblems can be recognized.
6.5 Backtracking
Backtracking explores a choice, abandons it when it cannot lead to a valid solution, and then tries another choice. It follows recursion, search spaces, and state because those concepts explain the call tree, the candidates being removed, and the information that must be restored.
7. Learn General Algorithm Design Strategies
Brute force makes the full work visible. Now learn three broad ways to reduce it: divide the problem into independent parts, commit to locally safe choices, or reuse answers to repeated states.
7.1 Divide and Conquer
Divide and conquer splits a problem into smaller mostly independent parts, solves them recursively, and combines their results. It follows recursion and differs from dynamic programming because the subproblems normally do not overlap substantially.
7.2 Merge Sort
Merge sort divides a sequence, sorts both halves, and merges them. It is the first full divide-and-conquer example and provides predictable O(n log n) time at the cost of additional merge storage.
7.3 Quick Sort
Quick sort partitions values around a pivot and recursively sorts the resulting parts. Study it after merge sort to compare two algorithms in the same design family whose memory use and average- and worst-case behavior differ.
7.4 Greedy
A greedy algorithm commits to the best-looking choice available at each step. It belongs after correctness because a local choice is useful only when the problem’s structure proves that the choice cannot destroy the global optimum.
7.5 Dynamic Programming as an Idea
Dynamic programming solves a problem by defining states and reusing answers to overlapping subproblems. Learn the idea after recursion and state, but postpone detailed forms until the recurrence and evaluation order can be developed systematically.
8. Represent and Traverse Graphs
Graphs turn relationships and possible movements into a common structure. By this stage, the reader already has the queues, stacks, sets, recursion, and state needed to focus on graph-specific reasoning.
8.1 Graph
A graph represents objects and the relationships or transitions between them. Start with the model itself so roads, dependencies, networks, grids, and game states can be recognized as versions of the same problem shape.
8.2 Vertex and Edge
A vertex represents an object or state, and an edge represents a relationship or permitted transition. These terms are the basic units used by every later graph representation and algorithm.
8.3 Directed and Undirected Graphs
A directed edge permits movement in one stated direction, while an undirected edge connects both ways. This distinction comes before traversal because reachability, cycles, components, and valid ordering all depend on direction.
8.4 Weighted Graph
A weighted graph attaches a cost, distance, capacity, or other value to each edge. Introduce weights before representation so adjacency structures can be read in both unweighted and weighted forms, even though shortest-path algorithms come later.
8.5 Adjacency List
An adjacency list stores each vertex with its outgoing or neighboring edges. It is the usual representation for sparse graphs and supports traversal in time proportional to the vertices and edges actually present.
8.6 Adjacency Matrix
An adjacency matrix stores edge information for every pair of vertices. Compare it with an adjacency list to see the trade-off between constant-time edge lookup and O(V^2) storage.
8.7 Visited Set
A visited set records vertices that have already been discovered or processed. It applies hash-set membership to prevent repeated work and nontermination when a graph contains multiple routes or cycles.
8.8 BFS
BFS uses a queue to visit vertices layer by layer from a start vertex. Learn it first because the queue order makes increasing unweighted distance visible and leads directly to unweighted shortest paths.
8.9 DFS
DFS uses recursion or an explicit stack to follow one path deeply before returning. Study it after BFS to compare how a change in the frontier structure changes traversal order and enables recursive graph reasoning.
8.10 Connected Components
Connected components partition an undirected graph into groups whose vertices can reach one another. Repeated BFS or DFS turns a single-start traversal into a classification of the whole graph.
8.11 Cycle Detection
Cycle detection determines whether traversal can return to an earlier vertex through a nontrivial route. It follows BFS and DFS because the exact test depends on traversal state and on whether the graph is directed or undirected.
8.12 Topological Sort
Topological sort orders a directed acyclic graph so every prerequisite appears before what depends on it. It comes after directed graphs and cycle detection because such an order exists only when no directed cycle is present.
9. Build Dynamic Programming Systematically
The earlier introduction established dynamic programming as state reuse. This stage turns that idea into a repeatable method by identifying repetition, writing a recurrence, choosing an evaluation direction, and then increasing the shape of the state.
9.1 Overlapping Subproblems
Overlapping subproblems occur when different computation paths ask for the same state more than once. Confirm this repetition first because storing results provides little benefit when subproblems are independent.
9.2 Recurrence
A recurrence defines a state’s answer from smaller or earlier states. It is the core of a dynamic programming solution because storage alone does not explain the transition, base cases, or dependency order.
9.3 Memoization
Memoization caches results as a recursive solution requests them. It is the first implementation style because it preserves the recurrence’s natural call structure while removing repeated evaluation.
9.4 Tabulation
Tabulation evaluates states iteratively in an order that guarantees their dependencies are already available. Study it after memoization to separate the recurrence itself from the choice between top-down and bottom-up execution.
9.5 One-Dimensional Dynamic Programming
One-dimensional dynamic programming uses one main index or quantity to identify a state. It is the first practical state shape because the table and transition order remain easy to trace.
9.6 Two-Dimensional Dynamic Programming
Two-dimensional dynamic programming uses two varying quantities to identify a state. It follows one-dimensional DP because adding a dimension increases both the number of states and the possible dependency directions.
9.7 Grid DP
Grid DP assigns a state to each cell and derives it from reachable neighboring cells. It is the first concrete two-dimensional form because position and transition direction can be visualized directly.
9.8 Knapsack
Knapsack problems choose items under a capacity or budget. They follow two-dimensional DP because the standard state combines progress through the items with the remaining or used capacity, then shows when one dimension can be compressed.
9.9 Longest Increasing Subsequence
Longest increasing subsequence asks for the longest subsequence whose values increase while original order is preserved. It consolidates sequence-state design and also provides a later opportunity to compare a direct DP solution with a faster binary-search-based method.
10. Solve Shortest-Path Problems
Shortest-path algorithms combine graph representation with different structural guarantees. Study them in increasing generality, and choose among them by edge cost, acyclicity, and whether the query is single-source or all-pairs.
10.1 BFS Shortest Path
BFS finds shortest paths when every edge has the same cost. It comes first because it reuses familiar traversal and makes clear why processing vertices in layers minimizes the number of edges.
10.2 DAG Shortest Path
DAG shortest path relaxes weighted edges in topological order. It follows topological sort and works even with negative weights because acyclicity guarantees that every predecessor can finish before its successors are processed.
10.3 Dijkstra
Dijkstra’s algorithm finds single-source shortest paths when every edge weight is non-negative. It combines greedy correctness with a priority queue, which repeatedly selects the cheapest unfinished route.
10.4 Bellman-Ford
Bellman-Ford repeatedly relaxes every edge and permits negative edge weights. Study it after Dijkstra to understand what is required when nonnegative-weight greedy reasoning no longer applies and how a reachable negative cycle can be detected.
10.5 Floyd-Warshall
Floyd-Warshall computes shortest paths between every pair of vertices by gradually allowing more intermediate vertices. It comes after single-source methods and basic DP because its three-dimensional recurrence explains the standard O(V^3) table update.
10.6 Path Reconstruction
Path reconstruction stores predecessor or next-step information so an optimal route can be recovered rather than only its cost. Learn it after distance algorithms so the additional state can be separated from the logic that computes the optimum.
11. Solve Connectivity and Spanning-Tree Problems
Traversal answers connectivity in a fixed graph. The next structures maintain changing groups and support algorithms that connect an entire weighted graph at minimum total cost.
11.1 Disjoint Set
A disjoint set is a collection of non-overlapping groups. Start with the abstraction so later operations have a clear meaning: determine which group contains an element and merge two groups.
11.2 Union-Find
Union-Find implements disjoint sets with find and union operations. It offers a non-traversal way to answer whether two elements are already connected while edges or relationships are added.
11.3 Path Compression
Path compression shortens parent chains during find. It comes after the basic structure because its purpose is to optimize repeated queries without changing the represented groups.
11.4 Union by Rank or Size
Union by rank or size attaches the shallower or smaller tree beneath the larger one. Combine it with path compression to keep the internal forest shallow and make operations nearly constant in amortized practice.
11.5 Minimum Spanning Tree
A minimum spanning tree connects every vertex of a connected undirected weighted graph with minimum total edge weight. Distinguish it from shortest path before learning an algorithm because it minimizes the cost of the whole connecting tree, not routes from one source.
11.6 Kruskal
Kruskal’s algorithm considers edges from lowest to highest weight and adds an edge when it joins two different components. It is the first spanning-tree algorithm because it directly combines greedy choice with Union-Find cycle prevention.
11.7 Prim
Prim’s algorithm grows one tree by repeatedly adding the cheapest edge leaving the current tree. Study it after Kruskal to compare an edge-ordered, component-based method with a frontier-based method using a priority queue.
12. Answer Repeated Range Queries
Prefix sums handle static cumulative queries well, but updates and other aggregation rules require richer structures. Learn this family by first naming the query, then matching the structure to whether data changes and what operation must be combined.
12.1 Range Query
A range query asks for a sum, minimum, maximum, or other aggregate over part of a sequence. Define the query and update pattern first because those constraints determine which structure is appropriate.
12.2 Fenwick Tree
A Fenwick tree supports prefix-based queries and point updates in logarithmic time. It is the first dynamic range structure because it extends the prefix-sum idea with a compact representation and relatively simple operations.
12.3 Segment Tree
A segment tree stores aggregates for nested intervals. It follows the Fenwick tree because it offers more flexible combine operations and query ranges at the cost of more storage and implementation complexity.
12.4 Lazy Propagation
Lazy propagation records deferred work inside a segment tree so range updates do not immediately visit every affected element. It must follow the basic segment tree because the delayed updates are meaningful only in relation to its interval nodes.
12.5 Sparse Table
A sparse table preprocesses an immutable sequence for fast range queries, especially idempotent operations such as minimum. Study it last to contrast an update-capable tree with a structure that gains speed by assuming the data never changes.
13. Add Mathematical Tools
These tools are not prerequisites for the first algorithmic patterns. They belong here because the reader can now connect each one to complexity reduction, preprocessing, recurrence, or state counting instead of treating mathematics as an isolated prerequisite block.
13.1 GCD
The greatest common divisor is the largest positive integer that divides two integers. The Euclidean algorithm makes it an early mathematical tool and provides another compact example of recursive reduction.
13.2 Modular Arithmetic
Modular arithmetic computes with remainders while preserving compatible addition and multiplication. It is needed when answers must stay within a modulus and prepares the reader for fast exponentiation, combinatorial counting, and rolling hashes.
13.3 Fast Exponentiation
Fast exponentiation computes a power by repeatedly squaring and using the binary form of the exponent. It applies divide-and-conquer reasoning to reduce a linear number of multiplications to logarithmic growth.
13.4 Sieve of Eratosthenes
The Sieve of Eratosthenes finds all primes up to a limit by marking multiples. It is a clear example of preprocessing many related queries together instead of testing each number independently.
13.5 Combinatorics
Combinatorics counts selections, arrangements, and partitions. Learn it before subset-based dynamic programming because it helps estimate the number of candidates and recognize when exhaustive enumeration is impossible.
13.6 Matrix Exponentiation
Matrix exponentiation applies fast exponentiation to a matrix that represents a linear state transition. It comes after recurrences, modular arithmetic, and fast exponentiation because it combines all three to advance a recurrence by many steps efficiently.
14. Exploit Structure in Strings
Strings are sequences, so earlier searching, hashing, trees, sorting, and preprocessing ideas now transfer to pattern problems. Begin with the general matching problem, then study structures that avoid repeated character comparisons.
14.1 String Matching
String matching asks where a pattern occurs inside a larger text. Start with the direct comparison method so the repeated work that later algorithms remove is visible.
14.2 KMP
KMP preprocesses the pattern’s prefix structure so a mismatch does not restart matching from the beginning. It is the first advanced matching algorithm because its saved work is deterministic and derived entirely from the pattern.
14.3 Rolling Hash
Rolling hash updates a substring hash when the window moves instead of recomputing it. Study it after modular arithmetic and KMP to compare probabilistic hash-based equality checks with deterministic prefix-based matching, including the risk of collisions.
14.4 Trie
A trie stores strings as paths that share prefix nodes. It applies the earlier tree model when many words must support prefix lookup, insertion, or dictionary traversal.
14.5 Suffix Array
A suffix array stores the starting positions of all suffixes in lexicographic order. It comes last because construction and use rely on sorting, binary search, and a mature understanding of prefix and substring relationships.
15. Extend Dynamic Programming to Complex States
Finish the path by changing the shape and representation of dynamic programming states. These forms rely on the earlier tree, graph, interval, combinatorial, and tabulation ideas rather than introducing dynamic programming again from the beginning.
15.1 Tree DP
Tree DP combines answers from child subtrees to compute a state for each node. It comes first because DFS supplies a natural dependency order and the absence of cycles makes state flow easier to reason about.
15.2 Interval DP
Interval DP defines a state for a contiguous range and derives it from smaller ranges. It follows two-dimensional DP and range techniques because the two endpoints identify the state while split points define the recurrence.
15.3 State Compression
State compression replaces a larger state description with a smaller representation that preserves every distinction needed by future transitions. Learn the principle before bitmask DP so the representation choice is justified by the information the algorithm must remember.
15.4 Bitmask DP
Bitmask DP represents a subset as bits and stores an answer for each relevant subset state. It is placed last because it combines search-space counting, state compression, combinatorics, and dynamic programming, and its exponential state count must be judged against the input limit.
The Main Point
The learning order begins with one visible algorithm, then adds the ability to measure cost, justify correctness, and reuse structure. Data structures appear before the algorithms that depend on them, and advanced families appear only after their shared design ideas are established.
Progress does not require mastering every variation before continuing. A topic is ready to support the next stage when the reader can state the problem it solves, trace its central mechanism, explain its main constraint, and recognize the kind of input for which it is useful.