Use junit platform runner to run junit 3, junit 4 and TestNG tests via vintage engine and testng engine - #3179

@olamy I did not see any positiv contribution from your site, only removals in favor of JUnit5.
The JUnit5 had one tendency 6 years ago, it was removal of Surefire and acceptance of "their" plugin. I know it because we talked about it that time.
If you follow this approach, you would break users, you would break the default behavior.
If you think tat Surefire Provider JUnit4 and Surefire Provider JUnit4.7+ are only about the version, then you do NOT understand this project.
If you think that Surefire Provider JUnit3 is only about JUnit3 then you again do not understand the point.
I saw people who remove something because they think that the old must go away but it happens because they are maybe corrupted or they do not want to understand, they do not listen and nothing exists for them only their ambition.
So, better for you is to collaborate with people who have long experiences with this project, who have the continuity of history and not to waste the energy with commits because they are really useless.
On the sire of Surefire there was a plan and the plan had a purpose but somebody did not understand it and deleted it. Without this plan some bugs cannot be fixed. Yesterday I talked to one contributor, he made a PR few days ago, and he understood point within several minutes but it is very hard to understand for someone who want's to do arrogant commits ignoring what I am telling you some weeks.

The JUnit5 had one tendency 6 years ago, it was removal of Surefire and acceptance of "their" plugin. I know it because we talked about it that time.

I assume you mean the "JUnit5 team"; the people behind the project. Right?

If yes, we had "that time" and still have today more than "one tendency". We provided and still provide help, options, and alternatives for many tool authors and vendors to adopt to the (then new) JUnit Platform. Nothing changed there, we still do it happily and will do so in the foreseeable future.

Are you mixing up the junit-platform-surefire-provider provided by the team (to support Surefire!) and my junit-platform-maven-plugin which was attempt to create Maven plugin talking directly to the JUnit Platform and playing nice with Java modules?

I saw people who remove something because they think that the old must go away but it happens because they are maybe corrupted or they do not want to understand, they do not listen and nothing exists for them only their ambition.

"corrupted"? The prefixed "maybe" doesn't make it sound any better.

While all participants should adhere to this code of conduct, we recognize that sometimes people may have a bad day, or be unaware of some of the code's guidelines. When that happens, you may reply to them and point out this code of conduct.

https://www.apache.org/foundation/policies/conduct

@Tibor17 : it has been reminded a few times that peoples who need JUnit 3 can stay on old versions of Surefire plugin. I have not seen a counter-argument to that. I have not seen neither a counter-argument to the fact that supporting JUnit 3 is an impediment to the development of new features. To recap:

• Why users who want to stay on old JUnit versions could not stay also on old Surefire versions?
• Considering that removing JUnit 3 support is necessary for making improvements easier, why supporting JUnit 3 in latest Surefire is more important than allowing the development of new features (multi-release support, etc.)?

Please note Junit3 code can be still executed but need the junit 4.12 dependency at least

Just to point out: according to the JUnit 6 release notes, the JUnit Vintage engine is deprecated and (to quote) "it should only be used temporarily while migrating tests to JUnit Jupiter or another testing framework with native JUnit Platform support".

What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?

Just to point out: according to the JUnit 6 release notes, the JUnit Vintage engine is deprecated and (to quote) "it should only be used temporarily while migrating tests to JUnit Jupiter or another testing framework with native JUnit Platform support".

What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?

You will have a few ways to go about this:

• Option 1: Easiest one, just stick with the last Surefire version before this PR is merged by setting it in your pluginManagement. Since you’re already on the older JUnit 3/4 (which are deprecated anyway), it’s totally fine to stay on an older Surefire for now.

• Option 2: Surefire will pick up whatever JUnit version or platform it finds (JUnit, JUnit Platform, or JUnit Vintage) from your pom. So unless you’re moving to JUnit 6.x (or a Vintage version that dropped JUnit 3/4 support), you should be all good.

• Option 3: If you’re thinking about updating things, you could use a tool like OpenRewrite or even an AI helper to migrate your tests to JUnit 5 or 6.

What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?

It's deprecated, but not for removal. Not slowly, not abruptly, not "everly".

We think of it more as we do of JUnit 3 and 4: they are here to stay. Until the last user vanishes. So does the glue-code in junit-vintage-engine. As long as the JUnit Platform is supported by tools (including Maven), you get all the integration with your testing framework of choice. Through an implementation of a test engine. And in case of JUnit 3 and 4, both projects are in an end-of-life status, no change is to be expected on their side.

Not discussed with the team, yet, but I could think of moving the junit-vintage-engine module as sibling right next to the testng-engine project, making it a standalone top-level project.

TestNG contributor here – and apologies if this isn't the right place to discuss this.

I believe this move could be positive for the Java community. While JUnit versions prior to 5 and TestNG are largely in maintenance mode today, the JUnit 5 platform has clearly become the most mature and actively maintained solution for testing in the Java ecosystem. From that perspective, moving the abstraction level from Maven to JUnit itself does make sense.

That said, I think @Tibor17 's concerns and experience are important. For a change of this importance, a more incremental approach may be worth considering if there is a risk of breaking existing setups. Based on my understanding, the testng-engine does not yet cover all TestNG features and still sees relatively limited adoption, which may increase the migration risk.

For example, would it be possible to keep the JUnit 5 provider enabled by default while disabling legacy providers, but still allow running both side by side during a transition phase?

@juherr please note version 3.6.x (at least first alphas/betas/Mx whatever name we use) will be anything but only multi providers simplification (no new features at least for few version)
So either you upgrade to this version or you simply keep using the previous version 3.5.x.
Not sure to understand what sort of more incremental approach could be used?

regarding

For example, would it be possible to keep the JUnit 5 provider enabled by default while disabling legacy providers, but still allow running both side by side during a transition phase?

you will be able to eventually do something such

     <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.6.xxx</version>
        <dependencies>
          <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit3</artifactId>
            <version>3.5.4</version>
          </dependency>
        </dependencies>
     </plugin>

There will be some backward compatibility for the providers but maybe not for long.
But what would be the benefit for someone doing such configuration? (again if any issue just change back surefire plugin version)
If there is any issue with the testng-engine, I'm pretty sure the team there will be happy to fix and release any issue quickly.

@olamy Yes, that is exactly what I had in mind as a transitional step.

The idea is simply to allow users to upgrade Surefire while keeping a lightweight fallback option in case issues are discovered during the migration. I agree that rolling back to the previous plugin version is also a valid workaround, so this is more about flexibility than a hard requirement.

And yes, I fully agree regarding testng-engine and I’m confident the team would react quickly if issues are reported.

Thanks for the clarification, and looking forward to 3.6.x.