Dear End-2-End test environment, Hi E2E, You and I have a special relation. We know each other for some time now, we go way back. It was not love at first side, but we found a way to make it work. In the beginning you gave me a save feeling, the confirmation that what I did was right. Working with you gave me some confidence, I had the feeling we made each other better. Strong alone, but stronger together. But soon I found out that this was all fake, it didn't come from both sides. You betrayed me by working with others and disturbed my flow. It started to frustrate me and I felt betrayed. I found out I invested too much time in you and did not got the return of investment I expected. The things you told me where incorrect. I'm so glad we broke up and I don't have to work with you anymore. I know you try to seduce others, but I'm not coming back. I'm not afraid of you, I will fight you on every opportunity. I hate you and I hope more and more people will stop working with you. Roy
For those who don’t know me, I have a special love/hate relationship with End-2-End test environments. I hate the environments and their setup, but I love the discussions about it. In an assignment at a big Dutch bank I had the opportunity to enable others to move away from E2E test environments. By not providing critical parts of these environments to other teams. This caused a lot of discussions and nice conversations. Last Devoxx (2019) I gave a talk about it: End-2-End test environments a dead End road In this blogpost series I will take the opportunity to share more about this topic.
In this first blog post I want to dive deeper into the definition, what is an end-2-end test environment. I know that E2E tests are known to be tests that do integration tests between frontend and backend. But that’s not what I mean when talking about E2E test environments. To me an E2E test environment is:
A full blown, integrated, production like environment that is used for the single purpose of testing.
So a setup, where every team and person puts his production like application and services. Connected to each other with the only purpose of testing. Deploying it in an E2E test environment is only done for testing or enabling testing. No stubs, no mocks. A complete set of functionalities is available to be tested. Some characteristics for these environments:
- Cross teams
- No stubs/mocks
- Contains only deployments that are also in production (with different configuration)
- Runs with test-data
Apparently these kind of environments have a lot of different names, as a real fan I’m trying to collect those names:
- Chain Environment
- User Acceptance Environment
- Production Acceptance Environment
- Broad-stack Environment (Martin Fowler)
- NGA Environment
- Retailer Test Environment
- Acceptance Environment
- Pre Production Environment
- Enterprise-wide Integration Test Environment
- Full-stack Environment
- Production-like Environment
- Fake news Environment
Most of the time these kind of environments have only 1 requirement:
It should be as production like as possible
A lot of companies have these kind of environments, it is flaky and often broken, but still people invest in it. Apparently people still need it? Or maybe no effort is put in changing the way we test our software? Can’t we say goodbye to the ‘A’ in DTAP (Develop, Test, Acceptance, Production)? I heard many arguments (next blog posts) to keep these kind of environments, but none of them hold ground. When the idea of abandoning the E2E Test environments formed, I was a bit skeptical (in the beginning). But I saw it is possible to work without it while achieving higher quality software and a more efficient team.
There is a lot of discussion about Architecture. Microservices, Monolith, Cloud, Serverless, etc. And switching to these architectures are done quite often. But the architectural questions are always about how production is going to run, not about how it should work for testing the software. By investing more in improving these setups, the team can be more efficient.
In the next blog post I will cover the reasons why I think these E2E test environments are bad.