Repository navigation
Smoke test the administrator APIs against each database - #570
Open
tonygermano wants to merge 4 commits into
Open
tonygermano wants to merge 4 commits into
tonygermano wants to merge 4 commits into
Conversation
run-harness.sh sorted every jar alphabetically, so the stock jersey-proxy-client-2.22.1.jar came before mirth-client-core.jar and mirth-server.jar. Both of those carry the engine's own copy of Jersey's WebResourceFactory, which adds multipart support, so the stock class won and every multipart API call from a harness test failed with "MessageBodyWriter not found ... multipart/form-data". The Client methods that send multipart requests, such as updateChannelGroups, could not be used from a smoke test. List server-lib/mirth-*.jar first, then the rest, without duplicates. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Tony Germano <tony@germano.name>
Waiting for the server to reach some state needs the same deadline, 500 ms sleep and retry loop each time. Add Harness.awaitAsserted, which runs a check until it passes or the harness timeout passes, and awaitAssertedValue, for a check that returns the value it verified. An AssertionError from the check means "not yet", and the check runs again. On timeout, a new AssertionError is thrown whose message is "Timed out after Ns: " followed by the last failure's message, with that failure's cause and stack trace, so the report says it timed out and still shows what the check last saw. A check throws FinalAssertionError for a failure that waiting can't fix. That and any other exception, such as a lost connection, are thrown at once. The value overload has its own name because a lambda that returns a value, such as () -> count == 0, would otherwise select it silently and pass without checking anything. Use it in OieServer.awaitStarted and Harness.runMessage. runMessage still stops as soon as every message is terminal, by throwing FinalAssertionError, and reports the same details. Some messages change: - A timeout while assertions fail now reads "Timed out after Ns: <fixture> failed: ...", where it used to leave out the timeout. - A timeout waiting for messages to be stored reads "Timed out after Ns: waiting for message(s) ...". - A failure on terminal messages is reported as Harness$FinalAssertionError. - A channel that doesn't start reports "Timed out after Ns: Channel ... has not started; its state is X" instead of "did not start within Ns; last state was X". Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Tony Germano <tony@germano.name>
tonygermano
requested review from
a team,
NicoPiel,
gibson9583,
jonbartels,
kayyagari,
kpalang,
mgaffigan and
ssrowe
November 10, 2026 07:55
Contributor
|
Can we remove the duplicative dead tests and awaitAsserted? Also: can you split the tests into topic? We should eventually have good coverage of each entity - and scripts and config maps have little to do with each other. |
Most smoke tests deploy a channel and send messages, and the server's tests of these controllers use mocks, so the SQL behind the administrator's other screens ran in no automated test. Add tests that drive those APIs through the client against the server and database under test: - UserAccountTest: create, update and remove a user, set and read preferences, and log in with a wrong and then a right password. - ChannelGroupTest, CodeTemplateLibraryTest, AlertTest, GlobalScriptsTest and ServerSettingsTest: save and reload a channel group, a code template library and template, an alert, the global scripts and a server setting, removing or restoring each afterwards. - ConfigurationMapTest: the same for the configuration map, which by default is stored in a file rather than the database, so it covers only the servlet. - EventBrowserTest: search audit events by name and attribute, and page through them. - MessageBrowserTest: search messages by content text, metadata column, custom metadata and attachment, page through them, reprocess one and remove some. - DataPrunerTest: run the pruner on a channel with pruning set and check the event it records. The channel's messages are too new to prune, so this covers the queries that select messages, not the deletes. - DebugUsageTest: deploy a channel with the debugger enabled and check the usage row that is reported and then deleted. updateChannelGroups replaces the whole set of groups, so the group test sends the existing groups with its own. Request bodies use mutable collections, because XStream rejects the JDK's immutable ones. Text searches use the stored case, because Oracle's are case-sensitive. The tests live in a new package, org.openintegrationengine.smoketest.admin, next to the generated .cases package, rather than among the harness classes. That makes SharedServer and OieServer public, along with the OieServer methods the tests call. OieServer also gains client(), newClient() and a deployChannel overload that takes a Channel and debug options. Deploying the same channel twice no longer records it twice for removal. Harness gains newChannel, which loads a channel fixture with a random id and a name that includes it, so tests never collide with each other or with a channel left by an earlier run. The tests wait on the server with Harness.awaitAsserted and awaitAssertedValue. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Tony Germano <tony@germano.name>
These classes are named *Tests, so the build never runs them, and the smoke tests now cover what they check against every database: - DonkeyEngineControllerTests starts and stops the engine and deploys a channel, but the deploy test's channel creation is commented out and it asserts nothing. Every smoke-test run starts the engine and deploys channels. - Donkey's MessageControllerTests creates and deletes messages, which every message the smoke tests send, and MessageBrowserTest's remove test, exercise through the API. DataPrunerTests and the server's MessageControllerTests stay. The smoke tests don't prune messages or run reprocessing and searches concurrently, so nothing covers what those check. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Tony Germano <tony@germano.name>
tonygermano
force-pushed
the
test/smoketest-admin-coverage
branch
from
November 11, 2026 12:53
620dd6d to
b6280a0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Most of the SQL behind the Administrator's screens doesn't run in any automated test. The CI fixtures under
ci/tests/deploy channels and send messages, and the server's tests of these controllers use mocks. The older database-backed tests (MessageControllerTests,DataPrunerTestsand others) are Derby-only and excluded from the build. A change to a SQL map or the database layer, such as the MyBatis and HikariCP upgrade in #545, could break user management, the event or message browser, or the pruner on one database without CI noticing.This PR adds smoke tests that drive those APIs through the client against the server and database under test, so they run on all seven CI configurations.
Commits
run-harness.shsorted every jar alphabetically, so the stockjersey-proxy-client-2.22.1.jarcame beforemirth-client-core.jarandmirth-server.jar, which carry the engine's ownWebResourceFactorywith multipart support. Every multipart call from a harness test failed with "MessageBodyWriter not found … multipart/form-data", so no smoke test could, for example, save channel groups. The engine'smirth-*.jarnow go first. This also lets the engine's override of a jTDS class take effect in the harness.Harness.awaitAssertedruns a check every 500 ms until it passes or the harness timeout passes.awaitAssertedValuedoes the same for a check that returns the value it verified:AssertionErrormeans "not yet", so the check runs again. On timeout, a newAssertionErroris thrown whose message starts with "Timed out after Ns:" followed by the last failure's message, so it says it timed out and still describes what the check saw.FinalAssertionErrorfor a failure that waiting can't fix. Any other exception, such as a lost connection, is thrown at once.() -> count == 0, would otherwise select it silently and pass without checking anything.OieServer.awaitStartedandHarness.runMessagenow use it.runMessagestill stops as soon as every message is terminal. A few failure messages change wording, mainly so that timeouts say they timed out. The commit message lists them, and nothing inci/orsmoketest/parses them.org.openintegrationengine.smoketest.admin, next to the generatedcasespackage, rather than among the harness classes.SharedServer,OieServerand the members the tests call are now public. A newHarness.newChannelloads a channel fixture with a random ID and a name that includes it, so tests never collide with each other or with a channel left by an earlier run. The tests useawaitAssertedandawaitAssertedValuewherever they wait on the server. The six test classes are:Limits
Every test leaves the server as it found it, so the tests can run in any order, or twice against the same server. The pruner test prunes every channel with pruning settings, so don't run it against a server whose data matters.
Testing
./gradlew clean buildpasses.Two agent reviews checked the commits, and the first also checked for overlap with existing tests. The only overlap is incidental: the shared admin login, the channel deploys that read code templates and global scripts, and
Harness.fetchMessagelooking up messages by ID. None of the writes, searches, paging, reprocessing, pruning or usage statistics were tested before.Drafted with Claude Code