The European AI Act
10 Things Software Teams Need to Know Now
Artificial intelligence has long become everyday reality.
It is embedded in search engines, recommendation systems, recruiting tools, diagnostic software, and an increasing number of business applications.
With the EU AI Act, Europe is now creating the first comprehensive legal framework for AI.
For many developers, this initially sounds like regulation. In reality, the AI Act is changing something far more fundamental:
It changes how we develop software.
AI governance, risk management, data quality, and monitoring will in future be part of the normal software lifecycle — just like performance, security, or usability.
Here are the 10 things every software team should understand now.
1. The AI Act Follows a Risk-Based Approach
The underlying principle is simple:
Not every AI carries the same risk.
The AI Act distinguishes four categories:
- Unacceptable Risk — prohibited
- High Risk — heavily regulated
- Limited Risk — transparency obligations
- Minimal Risk — barely regulated
Typical high-risk systems include AI in:
- Recruiting systems
- Credit decisions
- Medical diagnostics
- Critical infrastructure
For development teams this means:
The first step of every AI project is a risk classification.
2. High-Risk AI Requires a Quality Management System
High-risk systems must have a structured Quality Management System (QMS).
This includes:
- Documented development processes
- Risk management across the entire AI lifecycle
- Test and validation strategies
- Clear responsibilities
- Governance structures
Many of these requirements echo established standards such as:
- ISO 9001
- ISO/IEC 27001
- ISO/IEC 25010
The consequence:
AI compliance becomes part of software engineering governance.
3. Data Quality Becomes Regulatorily Relevant
AI is only as good as its training data.
The AI Act therefore requires clear data governance.
Training data must:
- Be representative
- Be as free as possible from bias
- Be documented
- Be gathered in a traceable way
This includes, among other things:
- Dataset documentation
- Data management practices
- Bias risk assessments
Flawed training data is therefore not just a quality problem.
It can be a regulatory risk.
4. Transparency Becomes Mandatory
Users must know when they are interacting with AI.
Examples:
- Chatbots must identify themselves as AI
- AI-generated content must be labelled
- Automated systems must be explainable
This affects many applications:
- Customer support bots
- Marketing automation
- Generative AI
Transparency thus becomes a UX and product design topic.
5. Human Oversight Remains Mandatory
The AI Act follows a clear principle:
AI may support — but not decide unsupervised.
Especially for high-risk systems, humans must be able to intervene.
For example in:
- Credit decisions
- Candidate selection
- Medical diagnoses
- Biometric identification
Software must therefore offer mechanisms to:
- Review decisions
- Stop systems
- Correct results
6. Logging and Traceability Are Mandatory
High-risk AI must be auditable.
For this, the AI Act requires, among other things:
- Logging of system decisions
- Documentation of model versions
- Traceability of training data
- Technical documentation of the architecture
In short:
AI traceability becomes the standard.
For dev teams this means:
MLOps, model versioning, and logging are in future not just best practice — but regulatory requirements.
7. AI Does Not End at Deployment
The AI Act considers AI across its entire lifecycle.
Companies must monitor systems during operation too.
This includes:
- Continuous monitoring
- Risk analysis in live operation
- Reporting of serious incidents
This principle is called post-market monitoring.
Or in other words:
Continuous AI quality assurance.
8. Foundation Models Get Their Own Rules
The AI Act also distinguishes General Purpose AI (GPAI).
This includes large foundation models that can be used in many applications.
Providers face additional requirements, such as:
- Documentation of training data
- Safety assessments
- Measures against misuse
Particularly capable models are classified as Systemic Risk AI Models.
9. Violations Are Expensive
The AI Act provides for high penalties.
Up to:
- €35 million
- 7% of global annual turnover
This puts the AI Act in a similar dimension to the GDPR.
AI compliance is therefore not just a technical topic — but a strategic risk management topic.
10. AI Governance Becomes Part of Modern Software Development
The most important effect is cultural.
AI systems must in future be developed with:
- AI risk management
- AI governance
- AI compliance
- Responsible AI
- AI quality assurance
This creates a new discipline:
AI Quality Engineering
For developers, testers, and architects this means:
Disciplines such as
- Risk-based testing
- Traceability
- Monitoring
- Quality management
suddenly become central to AI projects.
The 5 Biggest Misconceptions About the AI Act
1. "The AI Act only applies to AI companies."
Wrong. Even companies that only integrate AI into their software can be affected.
2. "This only concerns large corporations."
Startups and medium-sized software companies must also fulfil the requirements.
3. "The AI Act bans AI."
The AI Act prohibits very few applications — only certain forms of social scoring, for example.
4. "This only affects data scientists."
The AI Act affects:
- Product development
- Software architecture
- Testing
- Compliance
- Management
5. "That is still years away."
Many regulations are coming into force gradually from 2025 and 2026.
Conclusion
The EU AI Act will likely do for AI what the GDPR did for data.
A turning point.
Companies that engage early with
- AI governance
- AI risk management
- AI quality engineering
will be significantly better positioned in the long run.
Summary
- AI is regulated like critical software
- Risk classification becomes mandatory
- Data quality becomes regulatorily relevant
- Traceability, logging, and monitoring become standard
- AI governance becomes part of modern software development
