True A→Z → PhD-level Python Mastery Course Part2

Excellent.
We are now back on the original A→Z / PhD-level track, exactly where we should be.

This is PHASE 7.1 — Python Import System Internals.
If you understand this module deeply, you will never struggle with imports, circular dependencies, or project structure again.


📦 PHASE 7.1 — PYTHON IMPORT SYSTEM (INTERNALS, NOT SYNTAX)

This phase explains what actually happens when Python sees import x
— not just how to write imports, but how Python resolves, loads, caches, and executes modules.


7.1.1 WHAT IS A MODULE (REALLY)?

A module is simply:

✔ A Python file
✔ Loaded once
✔ Executed top to bottom
✔ Stored in memory as an object

import math

Internally:

  • math becomes a module object
  • Stored in memory
  • Reused everywhere
import math
print(type(math))   # <class 'module'>

📌 Interview line

A module is an object created by executing a Python file and storing its namespace.


7.1.2 IMPORT IS NOT JUST “ACCESS” — IT IS EXECUTION

This is critical.

# file: a.py
print("Hello from a")
x = 10
import a

Output:

Hello from a

What happened?

  • Python executed a.py
  • Created a namespace
  • Stored it as a module object

📌 Importing a module runs its code once


7.1.3 IMPORT EXECUTION FLOW (REAL PIPELINE)

When Python sees:

import mymodule

It does:

STEP 1 — Check cache

import sys
sys.modules

If mymodule is already in sys.modules:
✔ Return it immediately
✔ Do NOT re-execute the file


STEP 2 — Find the module (Search Order)

Python searches in this exact order:

  1. Built-in modules (sys, math)
  2. Current script directory
  3. Directories in PYTHONPATH
  4. Standard library
  5. Site-packages (installed libs)
import sys
print(sys.path)

📌 Interview killer

Import errors are path problems, not syntax problems.


STEP 3 — Load the module

  • Read source (.py)
  • Compile to bytecode (.pyc)
  • Execute top-level code
  • Create module object

STEP 4 — Cache it

Stored in:

sys.modules["mymodule"]

7.1.4 WHY IMPORT IS FAST AFTER FIRST TIME

import time
start = time.time()
import math
print(time.time() - start)

Almost zero.

WHY?

  • Already in sys.modules
  • No disk access
  • No execution

📌 Interview line

Python imports are idempotent due to module caching.


7.1.5 import x vs from x import y

Case 1

import module
  • Binds name module
  • Access via module.y

Case 2

from module import y
  • Binds name y directly
  • No module namespace

⚠️ Danger

from module import *

WHY?

  • Pollutes namespace
  • Hides origins
  • Breaks readability & tooling

📌 Interview rule

Never use import * outside REPL.


7.1.6 IMPORT BINDING ≠ OBJECT COPYING

# a.py
x = 10
from a import x
x = 20

Does this modify a.x?
❌ NO.

Because:

  • x is a new binding
  • Not a reference to the name

But:

import a
a.x = 20

✔ Modifies module state

📌 Key idea

Imports bind names, not variables.


7.1.7 MODULES ARE SINGLETONS (IMPORTANT)

import a
import a

✔ Same object
✔ Same memory
✔ Same state

import sys
sys.modules["a"]

This is why:

  • Global state persists
  • Configuration modules work
  • Circular imports occur

7.1.8 CIRCULAR IMPORT — THE REAL ROOT CAUSE

Example

# a.py
import b
x = 10
# b.py
import a
print(a.x)

❌ May fail with:

AttributeError: module 'a' has no attribute 'x'

WHY?

Import timeline:

  1. a starts loading
  2. a imports b
  3. b imports a
  4. a exists in sys.modules but not fully executed
  5. x not yet defined

📌 Interview line

Circular imports fail because modules are partially initialized.


7.1.9 HOW TO FIX CIRCULAR IMPORTS (PROPERLY)

❌ Bad fix

  • Reordering imports randomly

✅ Correct strategies

Strategy 1 — Move import inside function

def f():
    import b

✔ Delays import until runtime


Strategy 2 — Refactor shared logic

# common.py

Both modules import from common


Strategy 3 — Import module, not names

import a
a.x

7.1.10 __name__ == "__main__" (CRITICAL)

print(__name__)
  • If executed directly → "__main__"
  • If imported → module name

Pattern

if __name__ == "__main__":
    main()

📌 Used to:

  • Prevent execution on import
  • Write reusable scripts
  • Build CLIs

7.1.11 RUNNING A MODULE VS IMPORTING IT

python file.py

vs

python -m package.module

Difference:

  • -m runs module as part of a package
  • Fixes relative imports
  • Used by frameworks & tools

📌 Senior-level knowledge


7.1.12 BYTECODE & __pycache__

When importing:

  • Python creates .pyc
  • Stored in __pycache__
  • Used if source unchanged

Why it exists:
✔ Faster startup
✔ No recompilation


🧪 PRACTICE (DO NOT SKIP)

Q1 — Predict output

# a.py
print("A loaded")

# main.py
import a
import a

Q2 — Why does this fail?

from a import x
x = 20

Q3 — Fix circular import (design question)


Q4 — Why is import * dangerous?


🎯 INTERVIEW CHECKPOINT (MANDATORY)

You must now be able to explain:

✅ How Python resolves imports
✅ What sys.modules is
✅ Why imports are cached
✅ Why circular imports fail
✅ Difference between import x and from x import y
✅ Why __main__ exists
✅ How .pyc works

If you can explain these cleanly, you are already ahead of many senior developers.


🚀 NEXT STEP (STRICT CONTINUATION)

Reply with ONLY ONE number:

1 → PHASE 7.2: Packages, __init__.py, absolute vs relative imports, project layout
2 → Quick interview drill (15 tricky import-system questions)

We continue without skipping anything.

Perfect.
This is PHASE 7.2 — Packages, __init__.py, Absolute vs Relative Imports, Project Layout.

This phase is where many strong coders fail as engineers.
They can write code, but they cannot structure systems.
After this, you’ll understand how real Python codebases are built.


📦 PHASE 7.2 — PACKAGES & PROJECT STRUCTURE (DEEP, INTERVIEW-GRADE)


7.2.1 WHAT IS A PACKAGE (REALLY)?

A package is:

A directory that Python treats as a namespace of modules

Historically:

  • A directory with __init__.py → package

Modern Python (3.3+):

  • Namespace packages may exist without __init__.py
  • But real projects still use it deliberately

📌 Interview line

A package is a module that can contain other modules.


7.2.2 WHY __init__.py EXISTS (NOT JUST A MARKER)

❌ Wrong understanding

__init__.py only tells Python this is a package”

✅ Correct understanding

__init__.py is:

  • Executed on package import
  • Controls package API
  • Can run initialization logic
  • Can expose / hide submodules

Example

mypkg/
│
├── __init__.py
├── a.py
├── b.py
import mypkg

Python:

  1. Loads mypkg/__init__.py
  2. Creates package object
  3. Executes __init__.py

7.2.3 PACKAGE OBJECT IS A REAL OBJECT

import mypkg
print(type(mypkg))

Output:

<class 'module'>

✔ Packages are modules with a __path__ attribute

print(mypkg.__path__)

7.2.4 CONTROLLING THE PUBLIC API (__all__)

Inside mypkg/__init__.py:

__all__ = ["a"]

Effect:

from mypkg import *

✔ Only imports a

📌 Important

  • __all__ only affects import *
  • It does NOT prevent direct imports

7.2.5 GOOD vs BAD __init__.py USAGE

❌ BAD (side effects)

# __init__.py
print("Initializing package")
connect_to_database()

Why bad:

  • Runs on import
  • Unexpected behavior
  • Slows startup

✅ GOOD (API control)

# __init__.py
from .service import Service
from .errors import AppError

Now users can do:

from mypkg import Service

📌 Interview line

__init__.py defines the public surface of a package.


7.2.6 ABSOLUTE IMPORTS (RECOMMENDED)

from mypkg.utils.helpers import func

Why absolute imports are preferred

✔ Clear
✔ Unambiguous
✔ IDE-friendly
✔ Refactor-safe

📌 Official Python recommendation (PEP 328 / 328)


7.2.7 RELATIVE IMPORTS (ONLY INSIDE PACKAGES)

from . import utils
from ..core import engine

Rules:

  • Only work inside packages
  • Fail if module is run directly
  • Based on package hierarchy

📌 Relative imports are relative to package, not filesystem.


7.2.8 WHY RELATIVE IMPORTS BREAK WHEN RUN DIRECTLY

# mypkg/a.py
from .b import x

If you run:

python a.py

❌ ERROR:

ImportError: attempted relative import with no known parent package

WHY?

  • a.py is not running as part of a package
  • No package context

✅ Correct way to run

python -m mypkg.a

📌 Interview line

Relative imports only work when code is executed as a module.


7.2.9 ABSOLUTE vs RELATIVE — INTERVIEW DECISION TABLE

ScenarioUse
Library / frameworkAbsolute
Internal package refRelative (carefully)
ScriptsAbsolute
Public APIsAbsolute

📌 Rule of thumb

Prefer absolute imports; use relative only for tightly-coupled internals.


7.2.10 PROJECT LAYOUT — REAL-WORLD STANDARD

❌ BAD (common beginner layout)

project/
├── a.py
├── b.py
├── utils.py
├── main.py

Problems:

  • Flat namespace
  • Import conflicts
  • Unscalable

✅ GOOD (production layout)

project/
├── pyproject.toml
├── README.md
├── src/
│   └── myapp/
│       ├── __init__.py
│       ├── core/
│       │   ├── __init__.py
│       │   └── engine.py
│       ├── services/
│       │   ├── __init__.py
│       │   └── user.py
│       └── cli.py
└── tests/
    └── test_engine.py

📌 Benefits:
✔ Clear boundaries
✔ No import ambiguity
✔ Test-friendly
✔ Package-ready


7.2.11 WHY src/ LAYOUT IS IMPORTANT (INTERVIEW FAVORITE)

Without src/:

  • Python may import local files instead of installed package
  • Hidden bugs in production

With src/:

  • Forces correct import resolution
  • Mirrors installed behavior

📌 Senior-level insight


7.2.12 sys.path & PROJECT ROOT CONFUSION

Python decides imports using:

import sys
print(sys.path)

First entry:

  • Directory of the script being run

📌 This explains:

  • “Works locally but not in prod”
  • Test import issues
  • IDE vs CLI mismatches

7.2.13 COMMON INTERVIEW BUGS (REAL)

Bug 1 — Shadowing stdlib

json.py

Then:

import json

❌ Imports your file, not stdlib


Bug 2 — Circular package imports

services import models
models import services

❌ Partial initialization

Fix:

  • Move shared logic
  • Import inside functions
  • Redesign boundaries

7.2.14 EXECUTABLE PACKAGES

If you add:

mypkg/
├── __init__.py
├── __main__.py

You can run:

python -m mypkg

📌 Used by:

  • pip
  • pytest
  • black
  • uvicorn

🧪 PRACTICE (MANDATORY)

Q1

Why does this break?

from .utils import helper

when run directly?


Q2

What belongs in __init__.py and what should not?


Q3

Why is src/ layout safer?


Q4

Design a clean package structure for:

  • CLI tool
  • Internal services
  • Public API

🎯 INTERVIEW CHECKPOINT

You must now clearly explain:

✅ What a package is
✅ What __init__.py actually does
✅ How public APIs are exposed
✅ Absolute vs relative imports
✅ Why python -m matters
✅ Why src/ layout is used
✅ How imports break in real projects

If you can do this cleanly → you think like a Python engineer, not a script writer.


🚀 NEXT STEP (STRICT CONTINUATION)

Reply with ONLY ONE number:

1 → PHASE 7.3: Advanced import patterns (dynamic imports, plugins, lazy imports)
2 → 20-question import-system interview drill

We continue without deviation.

Excellent.
This is PHASE 7.3 — Advanced Import Patterns.
This phase explains how real frameworks, CLIs, plugin systems, and large codebases work.

If Phase 7.1–7.2 taught you how imports work,
Phase 7.3 teaches you how engineers exploit the import system intentionally.

No syntax tricks — design patterns + internals + interview-grade reasoning.


📦 PHASE 7.3 — ADVANCED IMPORT PATTERNS (DEEP)


7.3.1 DYNAMIC IMPORTS — IMPORT AT RUNTIME

What is a dynamic import?

Importing a module based on runtime information, not static code.

module_name = "math"
module = __import__(module_name)
print(module.sqrt(16))

But do not use __import__ directly in real code.


✅ Correct way — importlib

import importlib

math = importlib.import_module("math")
print(math.sqrt(16))

📌 Interview line

importlib is the official API for dynamic imports.


When dynamic imports are used

✔ Plugin systems
✔ Optional dependencies
✔ Feature flags
✔ Avoiding circular imports
✔ Reducing startup time


7.3.2 DYNAMIC IMPORT — REAL USE CASE

Example: Conditional backend selection

def get_db(backend):
    if backend == "postgres":
        return importlib.import_module("db.postgres")
    elif backend == "mysql":
        return importlib.import_module("db.mysql")

Why this matters:

  • Avoids importing unused modules
  • Reduces memory footprint
  • Faster startup

7.3.3 LAZY IMPORTS — IMPORT ONLY WHEN NEEDED

Problem

Heavy modules slow startup.

import pandas  # expensive

Solution — lazy import

def process():
    import pandas as pd
    return pd.DataFrame(...)

📌 Import happens only if function is called.


Interview-grade explanation

Lazy imports delay cost until the functionality is actually used.

Used heavily in:

  • Django
  • NumPy
  • CLI tools
  • ML frameworks

7.3.4 PERFORMANCE BENEFITS OF LAZY IMPORTS

AspectBenefit
Startup timeFaster
MemoryLower
Optional depsSafe
CLI toolsResponsive

7.3.5 PLUGIN SYSTEMS — CORE ADVANCED PATTERN

Goal

Allow users to add functionality without modifying core code.


Pattern 1 — Registry + Decorator (Most Common)

PLUGINS = {}

def register(name):
    def decorator(func):
        PLUGINS[name] = func
        return func
    return decorator

Plugin:

@register("json")
def json_parser(data):
    ...

Usage:

PLUGINS["json"](data)

📌 Used in:

  • ETL frameworks
  • Command systems
  • Task runners

7.3.6 PLUGIN DISCOVERY VIA DYNAMIC IMPORT

Directory structure:

plugins/
├── json_plugin.py
├── csv_plugin.py

Loader:

import importlib
import pkgutil
import plugins

def load_plugins():
    for _, module_name, _ in pkgutil.iter_modules(plugins.__path__):
        importlib.import_module(f"plugins.{module_name}")

What happens:

  • Modules are imported
  • Decorators register plugins
  • Core system remains unchanged

🔥 Framework-level design


7.3.7 HOW FRAMEWORKS AUTO-DISCOVER CODE

This is how:

  • pytest finds tests
  • Django loads apps
  • Click finds commands

📌 Interview line

Frameworks rely on import side effects intentionally.


7.3.8 IMPORT SIDE EFFECTS — POWERFUL BUT DANGEROUS

# plugin.py
print("Registered plugin")

On import:
✔ Code runs
❌ Side effects happen

Best practice

  • Only registration logic
  • No heavy computation
  • No IO

7.3.9 ENTRY POINTS (ADVANCED, REAL-WORLD)

Used by:

  • pip
  • pytest
  • black
  • flake8

Defined in pyproject.toml / setup.py

Concept:

External packages can register plugins without being imported explicitly

📌 Mention this in senior interviews, not junior ones.


7.3.10 CONDITIONAL IMPORTS (OPTIONAL DEPENDENCIES)

try:
    import numpy as np
except ImportError:
    np = None

Used for:
✔ Optional features
✔ Graceful degradation


7.3.11 AVOIDING CIRCULAR IMPORTS USING LAZY IMPORT

# a.py
def f():
    from b import g
    g()

✔ Import only when needed
✔ Avoids partial initialization


7.3.12 RELOADING MODULES (RARE, BUT ASKED)

import importlib
import mymodule

importlib.reload(mymodule)

⚠️ Dangers:

  • State inconsistencies
  • Multiple class definitions
  • Broken references

📌 Mostly for:

  • REPL
  • Development
  • Hot reload systems

7.3.13 IMPORT HOOKS (MENTION-LEVEL ONLY)

Python allows custom importers via:

  • sys.meta_path
  • importlib.abc.Loader

Used by:

  • Zip imports
  • Frozen apps
  • Custom runtimes

📌 Interview rule

Know they exist; don’t implement unless required.


7.3.14 COMMON INTERVIEW TRAPS (ADVANCED)

Trap 1 — Import inside loop

for _ in range(1000):
    import math

✔ Safe
✔ Cached
❌ Confusing — avoid stylistically


Trap 2 — Assuming import order

import a
import b

❌ Order may matter if side effects exist


Trap 3 — Plugin import failure

One bad plugin → crashes all
✔ Solution: isolate import errors


🧪 PRACTICE (INTERVIEW-LEVEL)

Q1

Why are dynamic imports useful in large systems?


Q2

Explain how a plugin system works using imports.


Q3

Why do frameworks rely on import side effects?


Q4

When would you avoid lazy imports?


Q5

Why is importlib preferred over __import__?


🎯 PHASE 7 — FINAL INTERVIEW CHECKPOINT

You must now confidently explain:

✅ How dynamic imports work
✅ Lazy imports and performance trade-offs
✅ Plugin discovery patterns
✅ Import side effects (good vs bad)
✅ How frameworks auto-register code
✅ Optional dependencies handling
✅ Reloading and why it’s dangerous

If you can explain these clearly and calmly, you are senior/staff-level in Python import systems.


🏁 PHASE 7 COMPLETE ✅

You now fully understand:
✔ Import resolution
✔ Module caching
✔ Packages & layouts
✔ Advanced import patterns

This is rare knowledge, even among experienced developers.


🚀 NEXT PHASE (STRICT A→Z CONTINUATION)

Reply with ONLY ONE number:

1 → PHASE 8: Exceptions & Error Handling (deep, production-grade)
2 → 30-question import-system interview drill (brutal)

We continue without deviation.