Skip to main content

Over-Abstracting: Too Many Layers

Adding abstraction layers beyond what's needed, reducing clarity.

TL;DR​

Over-abstracting creates unnecessary indirection that makes code harder to follow without providing real benefits. A request passes through 8+ layers before doing anything useful. Simple tasks require understanding 15 files. Speculative abstractions "just in case" we need them later add complexity without solving current problems. The solution: use abstraction only when it solves an actual problem, keep architectures flat (2-3 layers), and resist the urge to abstract prematurely.

Learning Objectives​

You will be able to:

  • Identify when abstraction adds value vs. when it adds complexity
  • Design layered architectures with minimal indirection
  • Understand the costs of each abstraction layer
  • Refactor over-abstracted code back to simplicity
  • Apply YAGNI (You Aren't Gonna Need It) principle to architecture
  • Balance flexibility with understandability

Motivating Scenario​

You need to fetch a user's data and render it. In an over-abstracted codebase:

1. HTTP request arrives at Controller
2. Controller calls Handler
3. Handler calls Manager
4. Manager calls Service
5. Service calls Repository
6. Repository calls Mapper
7. Mapper calls Converter
8. Converter calls the Database
9. Returns back through 7 layers

Following this code path requires opening 15 files. Each layer does almost nothing—passes data, transforms it slightly, and passes it to the next layer. This is indirection without purpose.

Debugging is a nightmare. User says "the data is wrong." You trace through 8 layers. Each layer looks correct. By the time you reach the database, you've lost track of the original bug.

A simpler design:

1. Controller calls Service
2. Service queries Repository
3. Repository calls Database

Same functionality, 3 files instead of 15, clear data flow.

Core Explanation​

Abstraction Has a Cost

Each layer adds:

  • Mental Overhead: One more thing to understand
  • Indirection: Code path is harder to follow
  • Testing Complexity: More mocks, more test setup
  • Performance: Additional function calls (negligible but not zero)
  • Maintenance: Another place to change when requirements evolve

Abstraction only pays for itself if the benefit exceeds the cost.

When Abstraction Adds Value

  1. Code Reuse: Same logic needed in 3+ places
  2. Complexity Isolation: A complex concern deserves its own layer
  3. Testing: Mocking dependencies improves testability
  4. Future Flexibility: Likely future changes (based on actual requirements, not speculation)

Speculative Abstraction

"We might need a Converter layer someday." This is speculation. Speculative abstraction:

  • Solves problems that don't exist yet
  • Adds complexity to the current system
  • Often wrong (you don't actually need it later, or you need something different)
  • Delays actual feature work

Rule: Don't abstract until you have 2-3 real use cases demanding it.

Pattern Visualization​

Over-Abstraction vs. Appropriate Abstraction

Code Examples​

user_module.py
# usercontroller.py
class UserController:
    def __init__(self):
        self.handler = UserHandler()

    def get_user(self, user_id):
        return self.handler.handle_get_user(user_id)

# userhandler.py
class UserHandler:
    def __init__(self):
        self.manager = UserManager()

    def handle_get_user(self, user_id):
        return self.manager.manage_get_user(user_id)

# usermanager.py
class UserManager:
    def __init__(self):
        self.service = UserService()

    def manage_get_user(self, user_id):
        return self.service.get_user(user_id)

# userservice.py
class UserService:
    def __init__(self):
        self.processor = UserProcessor()

    def get_user(self, user_id):
        return self.processor.process_user_data(user_id)

# userprocessor.py
class UserProcessor:
    def __init__(self):
        self.repository = UserRepository()
        self.mapper = UserMapper()

    def process_user_data(self, user_id):
        user = self.repository.get_user(user_id)
        return self.mapper.map_user(user)

# usermapper.py
class UserMapper:
    def __init__(self):
        self.converter = UserConverter()

    def map_user(self, user):
        return self.converter.convert(user)

# userconverter.py
class UserConverter:
    def convert(self, user):
        return {
            'id': user.id,
            'name': user.name,
            'email': user.email
        }

# userrepository.py
class UserRepository:
    def get_user(self, user_id):
        # Fetch from database
        return database.query('SELECT * FROM users WHERE id = ?', user_id)

# To fetch a user:
# Request → Controller → Handler → Manager → Service → Processor
#           → Mapper → Converter → Repository → Database
# That's 8 layers, each doing almost nothing!

Patterns and Pitfalls​

Why Over-Abstraction Happens​

1. Pattern Enthusiasm After learning design patterns, developers want to use them everywhere. "We should have a Manager layer!" without asking if we need one.

2. Enterprise Architecture Cargo Cult Copying large systems' architecture without understanding context. Big systems have many layers for good reasons (scaling, team distribution, legacy constraints).

3. Future-Proofing Anxiety "What if we need to swap the database?" Add an abstraction layer just in case. But you don't swap databases. The abstraction stays forever.

4. Avoiding Refactoring It feels easier to add a new layer than to refactor existing code. Layers accumulate.

The YAGNI Principle​

"You Aren't Gonna Need It"

Build what you need now, not what you might need someday. Resist speculative abstraction. If you later need abstraction, you can refactor.

The cost of adding abstraction later (when you know you need it) is lower than the cost of carrying unnecessary abstraction forever.

When This Happens / How to Detect​

Red Flags:

  1. A request flows through 5+ classes to do something simple
  2. You have classes like Handler, Manager, Processor that just pass data
  3. No one understands why a layer exists
  4. Tests require mocking 8+ dependencies for one assertion
  5. Adding a feature requires changes in 6+ files
  6. One layer has no business logic, just delegation
  7. There's a "converter" layer that does string-to-object conversion
  8. Comments like "This layer exists for future flexibility"

Measurement:

Count the layers for a simple user story:

Simple change: GET /users/123 and return JSON

Layers touched:
1. HttpHandler
2. RequestRouter
3. Controller
4. Handler
5. Manager
6. Service
7. DataProcessor
8. Repository
9. Database

Should be: Controller → Service → Repository

How to Fix / Refactor​

Step 1: Trace a Request​

Pick a simple user story and trace the code path:

controller.getUser(123)
  → handler.handleGetUser(123)
    → manager.manageGetUser(123)
      → service.getUser(123)
        → processor.processUser(123)
          → mapper.mapUser(...)
            → converter.convert(...)
              → repository.getUser(123)

Count the layers (8).

Step 2: Evaluate Each Layer​

Does this layer:

  • Solve a real problem?
  • Hide complexity?
  • Enable code reuse?
  • Enable testing?

If none of these, mark it for removal.

Step 3: Consolidate​

Merge related layers:

// Before
controller.getUser → handler.handleGetUser → manager.manageGetUser

// After
controller.getUser (handler and manager logic are inlined)

Step 4: Test After Each Change​

Ensure behavior is preserved. Integration tests verify the change is correct.

Operational Considerations​

Refactoring Legacy Over-Abstraction

Use the "bubble up" technique:

  1. The top layer directly calls the bottom layer (skipping middle layers)
  2. If tests still pass, the middle layer was unnecessary
  3. Remove it
  4. Repeat

Knowing When to Backfill Abstraction

When you have 3+ similar operations needing the same abstraction, create it. Not before.

Design Review Checklist​

  • Is the request path through 3 or fewer layers?
  • Does each layer have a clear, single responsibility?
  • Are there any layers that just pass data without transforming?
  • Can tests mock only the dependencies they need (< 3 mocks)?
  • Are all layers used by actual features, not speculative?
  • Is there duplicate business logic across layers?
  • Can a new developer understand the layer structure in 10 minutes?
  • Would removing any layer break functionality?
  • Are layers justified by actual requirements, not future speculation?
  • Is the architecture simpler than the problem it solves?

Showcase​

Signals of Over-Abstraction

Anti-Signals (Problems)
  • Request flows through 7+ layers
  • Classes like Handler, Manager, Processor doing delegation only
  • Layer exists 'for future flexibility'
  • Tests mock 10+ dependencies
  • Adding feature requires changes in 6+ files
  • Hard to explain what a layer does
Healthy Signals (Solutions)
  • Request flows through 2-3 layers
  • Each class/layer has clear responsibility
  • Layers solve actual, current problems
  • Tests mock 1-2 dependencies
  • Feature changes localized to 1-2 layers
  • Layers are easy to explain and understand

Self-Check​

  1. Can you explain what each layer does in one sentence? If a layer's purpose is vague, it might be unnecessary.

  2. How many layers does a GET request pass through? If > 3, likely over-abstracted.

  3. Would removing a layer break functionality or just make code less maintainable? If it would break things, it's necessary. If removing it just requires minor refactoring, it was unnecessary.

Next Steps​

  • Map: Trace a simple request through your codebase
  • Count: How many layers does it pass through?
  • Evaluate: Does each layer solve a problem?
  • Consolidate: Merge unnecessary layers
  • Test: Ensure behavior is unchanged after refactoring

One Takeaway​

ℹ️

Don't abstract for flexibility you don't yet need. Build what solves today's problem. When you have 2-3 real use cases, then abstract.

References​

  1. YAGNI: You Aren't Gonna Need It ↗️
  2. Extract Method Pattern ↗️
  3. Patterns of Distributed Systems ↗️
  4. Indirection in Software Design ↗️