Area 2.11

Common Algorithms

Master fundamental search and sort algorithms. Learn to analyze algorithm complexity, compare performance metrics, and choose appropriate algorithms for different scenarios.

5
Core Algorithms
4
Performance Metrics
~3hrs
Study Time

Learning Objectives

  • Understand and implement linear search and binary search algorithms
  • Apply bubble sort, insertion sort, and merge sort techniques
  • Analyze algorithm complexity in terms of time and space
  • Compare algorithm performance using Big O notation
  • Choose appropriate algorithms based on data size and requirements
  • Evaluate trade-offs between different algorithmic approaches

Search Algorithms

Linear Search

O(n)

Sequentially check each element until target is found

Advantages:

  • Works on unsorted data
  • Simple to implement
  • No preprocessing needed

Disadvantages:

  • Slow for large datasets
  • Must check every element in worst case

Best Used For:

Small datasets, unsorted data, simple searches

Python Implementation:

def linear_search(arr, target):
    """Search for target in array sequentially"""
    for i in range(len(arr)):
        if arr[i] == target:
            return i  # Return index where found
    return -1  # Not found

# Example usage
numbers = [64, 34, 25, 12, 22, 11, 90]
target = 22

result = linear_search(numbers, target)
if result != -1:
    print(f"Found {target} at index {result}")
else:
    print(f"{target} not found")

# Time complexity: O(n) - may need to check all elements
# Space complexity: O(1) - uses constant extra space

Binary Search

O(log n)

Efficiently search sorted data by repeatedly dividing search space in half

Advantages:

  • Very fast for large sorted datasets
  • Efficient divide-and-conquer approach

Disadvantages:

  • Requires sorted data
  • More complex to implement
  • Needs preprocessing for unsorted data

Best Used For:

Large sorted datasets, frequent searches, performance-critical applications

Python Implementation:

def binary_search(arr, target):
    """Search for target in sorted array using binary search"""
    left, right = 0, len(arr) - 1
    
    while left <= right:
        mid = (left + right) // 2
        
        if arr[mid] == target:
            return mid  # Found target
        elif arr[mid] < target:
            left = mid + 1  # Search right half
        else:
            right = mid - 1  # Search left half
    
    return -1  # Not found

# Example usage
sorted_numbers = [11, 12, 22, 25, 34, 64, 90]  # Must be sorted!
target = 25

result = binary_search(sorted_numbers, target)
if result != -1:
    print(f"Found {target} at index {result}")

# Time complexity: O(log n) - halves search space each step
# Space complexity: O(1) - uses constant extra space

Sort Algorithms

Bubble Sort

O(n²)

Repeatedly step through list, compare adjacent elements and swap if needed

Advantages:

  • Simple to understand and implement
  • Stable sort (maintains relative order)
  • In-place sorting

Disadvantages:

  • Very slow for large datasets
  • Poor average and worst-case performance

Best Used For:

Educational purposes, very small datasets, when simplicity is more important than speed

Python Implementation:

def bubble_sort(arr):
    """Sort array using bubble sort algorithm"""
    n = len(arr)
    
    # Traverse through all elements
    for i in range(n):
        swapped = False
        
        # Last i elements are already sorted
        for j in range(0, n - i - 1):
            # Swap if element is greater than next element
            if arr[j] > arr[j + 1]:
                arr[j], arr[j + 1] = arr[j + 1], arr[j]
                swapped = True
        
        # If no swapping happened, array is sorted
        if not swapped:
            break
    
    return arr

# Example usage
numbers = [64, 34, 25, 12, 22, 11, 90]
print("Original:", numbers)
sorted_numbers = bubble_sort(numbers.copy())
print("Sorted:", sorted_numbers)

# Time complexity: O(n²) worst/average case, O(n) best case
# Space complexity: O(1) - sorts in place

Insertion Sort

O(n²)

Build sorted array one element at a time by inserting elements in correct position

Advantages:

  • Efficient for small datasets
  • Adaptive (fast for nearly sorted data)
  • Stable and in-place

Disadvantages:

  • Slow for large datasets
  • Quadratic time complexity in worst case

Best Used For:

Small datasets, nearly sorted data, online algorithms (sorting data as it arrives)

Python Implementation:

def insertion_sort(arr):
    """Sort array using insertion sort algorithm"""
    
    # Traverse from second element to end
    for i in range(1, len(arr)):
        key = arr[i]  # Current element to insert
        j = i - 1
        
        # Move elements greater than key one position ahead
        while j >= 0 and arr[j] > key:
            arr[j + 1] = arr[j]
            j -= 1
        
        # Insert key at correct position
        arr[j + 1] = key
    
    return arr

# Example usage
numbers = [64, 34, 25, 12, 22, 11, 90]
print("Original:", numbers)
sorted_numbers = insertion_sort(numbers.copy())
print("Sorted:", sorted_numbers)

# Time complexity: O(n²) worst case, O(n) best case
# Space complexity: O(1) - sorts in place

Merge Sort

O(n log n)

Divide array into halves, sort each half, then merge sorted halves

Advantages:

  • Consistently fast performance
  • Stable sort
  • Predictable time complexity

Disadvantages:

  • Uses additional memory
  • More complex to implement
  • Slower for very small datasets

Best Used For:

Large datasets, when consistent performance is needed, external sorting

Python Implementation:

def merge_sort(arr):
    """Sort array using merge sort algorithm (divide and conquer)"""
    if len(arr) <= 1:
        return arr
    
    # Divide array into two halves
    mid = len(arr) // 2
    left_half = merge_sort(arr[:mid])
    right_half = merge_sort(arr[mid:])
    
    # Merge sorted halves
    return merge(left_half, right_half)

def merge(left, right):
    """Merge two sorted arrays into one sorted array"""
    result = []
    i = j = 0
    
    # Compare elements and merge in sorted order
    while i < len(left) and j < len(right):
        if left[i] <= right[j]:
            result.append(left[i])
            i += 1
        else:
            result.append(right[j])
            j += 1
    
    # Add remaining elements
    result.extend(left[i:])
    result.extend(right[j:])
    
    return result

# Example usage
numbers = [64, 34, 25, 12, 22, 11, 90]
sorted_numbers = merge_sort(numbers)
print("Sorted:", sorted_numbers)

# Time complexity: O(n log n) all cases
# Space complexity: O(n) - needs additional memory

Algorithm Complexity Comparison

AlgorithmBest CaseAverage CaseWorst CaseSpace
Linear SearchO(1)O(n)O(n)O(1)
Binary SearchO(1)O(log n)O(log n)O(1)
Bubble SortO(n)O(n²)O(n²)O(1)
Insertion SortO(n)O(n²)O(n²)O(1)
Merge SortO(n log n)O(n log n)O(n log n)O(n)

Performance Analysis Metrics

Time Complexity

How execution time grows with input size

Examples:

  • O(1): Constant time
  • O(n): Linear time
  • O(log n): Logarithmic time
  • O(n²): Quadratic time

Critical for performance with large datasets

Space Complexity

How memory usage grows with input size

Examples:

  • O(1): Constant space
  • O(n): Linear space
  • O(log n): Logarithmic space

Important when memory is limited

Stability

Whether equal elements maintain their relative order

Examples:

  • Stable: Merge sort, Insertion sort
  • Unstable: Quick sort (typically)

Matters when sorting objects with multiple fields

Adaptability

How well algorithm performs on nearly sorted data

Examples:

  • Adaptive: Insertion sort
  • Non-adaptive: Merge sort

Useful for maintaining sorted collections

Learning Activities

Algorithm Implementation Challenge

Implementation
90 minutes

Implement all search and sort algorithms from scratch and test with different datasets

Performance Comparison Experiment

Analysis
60 minutes

Measure and compare actual execution times for different algorithms with varying data sizes

Algorithm Selection Workshop

Decision
45 minutes

Given various scenarios, choose the most appropriate search or sort algorithm