Testing

./gradlew assemble

./gradlew :distribution:archives:linux-tar:assemble
./gradlew :distribution:archives:darwin-tar:assemble
./gradlew :distribution:archives:windows-zip:assemble

In order to run Elasticsearch from source without building a package, you can run it using Gradle:

Launching and debugging from an IDE

./gradlew run --debug-jvm

Distribution

./gradlew run -Drun.distribution=oss

License type

./gradlew run -Drun.license_type=trial

Other useful arguments

You can run a single test, provided that you specify the Gradle project. See the documentation on simple name pattern filtering.

Run a single test case in the server project:

Run all tests in a package and its sub-packages:

Run all tests that are waiting for a bugfix (disabled by default)

Run with a given seed (seed is a hex-encoded long).

Every test repetition will have a different method seed (derived from a single random master seed).

Every test repetition will have exactly the same master (0xdead) and method-level (0xbeef) seed.

(note the filters - individual test repetitions are given suffixes, ie: testFoo[0], testFoo[1], etc…​ so using testmethod or tests.method ending in a glob is necessary to ensure iterations are run).

Repeats N times but skips any tests after the first failure or M initial failures.

Test groups can be enabled or disabled (true/false).

Default value provided below in [brackets].

By default the tests run on multiple processes using all the available cores on all available CPUs. Not including hyper-threading. If you want to explicitly specify the number of JVMs you can do so on the command line:

Or in ~/.gradle/gradle.properties:

Its difficult to pick the "right" number here. Hypercores don’t count for CPU intensive tests and you should leave some slack for JVM-internal threads like the garbage collector. And you have to have enough RAM to handle each JVM.

It is possible to provide a version that allows to adapt the tests behaviour to older features or bugs that have been changed or fixed in the meantime.

Run all tests without stopping on errors (inspect log files).

Run more verbose output (slave JVM parameters, etc.).

Change the default suite timeout to 5 seconds for all tests (note the exclamation mark).

Change the logging level of ES (not Gradle)

Print all the logging output from the test runs to the commandline even if tests are passing.

Configure the heap size.

Pass arbitrary jvm arguments.

Pass build arguments.

./gradlew check

./gradlew internalClusterTest

./gradlew precommit

restResources {
  restApi {
    includeCore '_common', 'indices', 'index', 'cluster', 'nodes', 'get', 'ingest'
  }
}

./gradlew :rest-api-spec:yamlRestTest

./gradlew ':rest-api-spec:yamlRestTest' \
  --tests "org.elasticsearch.test.rest.ClientYamlTestSuiteIT" \
  -Dtests.method="test {p0=cat.segments/10_basic/Help}"

./gradlew :modules:mapper-extras:javaRestTest

./gradlew ':modules:mapper-extras:javaRestTest' \
  --tests "org.elasticsearch.index.mapper.TokenCountFieldMapperIntegrationIT.testSearchByTokenCount {storeCountedFields=true loadCountedFields=false}"

The packaging tests also support Windows Server 2012R2 and Windows Server 2016. Unfortunately we’re not able to provide boxes for them in open source use because of licensing issues. Any Virtualbox image that has WinRM and Powershell enabled for remote users should work.

Specify the image IDs of the Windows boxes to gradle with the following project properties. They can be set in ~/.gradle/gradle.properties like

or passed on the command line like -Pvagrant.windows-2012r2.id=my-image-id -Pvagrant.windows-2016=another-image-id

These properties are required for Windows support in all gradle tasks that handle packaging tests. Either or both may be specified.

If you’re running vagrant commands outside of gradle, specify the Windows boxes with the environment variables

• VAGRANT_WINDOWS_2012R2_BOX

• VAGRANT_WINDOWS_2016_BOX

It’s important to think of VMs like cattle. If they become lame you just shoot them and let vagrant reprovision them. Say you’ve hosed your precise VM:

All you’ve got to do to get another one is

The whole process takes a minute and a half on a modern laptop, two and a half without vagrant-cachier.

Its possible that some downloads will fail and it’ll be impossible to restart them. This is a bug in vagrant. See the instructions here for how to work around it: https://github.com/mitchellh/vagrant/issues/4479

Some vagrant commands will work on all VMs at once:

vagrant up would normally start all the VMs but we’ve prevented that because that’d consume a ton of ram.

Because our packaging tests are capable of testing many combinations of OS (e.g., Windows, Linux, etc.), package type (e.g., zip file, RPM, etc.), Elasticsearch distribution type (e.g., default or OSS), and so forth, it’s faster to develop against smaller subsets of the tests. For example, to run tests for the default archive distribution on Fedora 28:

These test tasks can use the --tests, --info, and --debug parameters just like non-OS tests can. For example:

./gradlew bwcTest

./gradlew v5.3.2#bwcTest

./gradlew :x-pack:qa:rolling-upgrade:v7.7.0#bwcTest \
 -Dtests.class=org.elasticsearch.upgrades.UpgradeClusterClientYamlTestSuiteIT \
 -Dtests.method="test {p0=*/40_ml_datafeed_crud/*}"

Sometimes a backward compatibility change spans two versions. A common case is a new functionality that needs a BWC bridge in an unreleased versioned of a release branch (for example, 5.x). To test the changes, you can instruct Gradle to build the BWC version from a another remote/branch combination instead of pulling the release branch from GitHub. You do so using the bwc.remote and bwc.refspec.BRANCH system properties:

The branch needs to be available on the remote that the BWC makes of the repository you run the tests from. Using the remote is a handy trick to make sure that a branch is available and is up to date in the case of multiple runs.

Example:

Say you need to make a change to master and have a BWC layer in 5.x. You will need to: . Create a branch called index_req_change off your remote ${remote}. This will contain your change. . Create a branch called index_req_bwc_5.x off 5.x. This will contain your bwc layer. . Push both branches to your remote repository. . Run the tests with ./gradlew check -Dbwc.remote=${remote} -Dbwc.refspec.5.x=index_req_bwc_5.x.

For some BWC testing scenarios, you want to use the local clone of the repository without fetching latest. For these use cases, you can set the system property tests.bwc.git_fetch_latest to false and the BWC builds will skip fetching the latest from the remote.

If the following limitations cannot be observed, or there is a need to actually test some use case that is not available/allowed in fips mode, the test can be muted. For unit tests or Java rest tests one can use

For specific YAML rest tests one can use

For disabling entire types of tests for subprojects, one can use for example:

in build.gradle.

The following should be taken into consideration when writing new tests or adjusting existing ones:

TLS

Password hashing algorithm

Password length

Keystore Password

There are multiple base classes for tests:

• ESTestCase: The base class of all tests. It is typically extended directly by unit tests.

• ESSingleNodeTestCase: This test case sets up a cluster that has a single node.

• ESIntegTestCase: An integration test case that creates a cluster that might have multiple nodes.

• ESRestTestCase: An integration tests that interacts with an external cluster via the REST API. This is used for Java based REST tests.

• ESClientYamlSuiteTestCase : A subclass of ESRestTestCase used to run YAML based REST tests.

What kind of tests should I write?

Refactor code to make it easier to test

Use randomized-testing for coverage

Abuse randomization in multi-threaded tests

Read your IDE documentation for how to attach a debugger to a JVM process.

== Building with extra plugins
Additional plugins may be built alongside elasticsearch, where their
dependency on elasticsearch will be substituted with the local elasticsearch
build. To add your plugin, create a directory called elasticsearch-extra as
a sibling of elasticsearch. Checkout your plugin underneath elasticsearch-extra
and the build will automatically pick it up. You can verify the plugin is
included as part of the build by checking the projects of the build.

== Environment misc

There is a known issue with macOS localhost resolve strategy that can cause
some integration tests to fail. This is because integration tests have timings
for cluster formation, discovery, etc. that can be exceeded if name resolution
takes a long time.
To fix this, make sure you have your computer name (as returned by `hostname`)
inside `/etc/hosts`, e.g.:
....
127.0.0.1       localhost ElasticMBP.local
255.255.255.255 broadcasthost
::1             localhost ElasticMBP.local`
....

== Benchmarking

For changes that might affect the performance characteristics of Elasticsearch
you should also run macrobenchmarks. We maintain a macrobenchmarking tool
called https://github.com/elastic/rally[Rally]
which you can use to measure the performance impact. It comes with a set of
default benchmarks that we also
https://elasticsearch-benchmarks.elastic.co/[run every night]. To get started,
please see https://esrally.readthedocs.io/en/stable/[Rally's documentation].

== Test doc builds

The Elasticsearch docs are in AsciiDoc format. You can test and build the docs
locally using the Elasticsearch documentation build process. See
https://github.com/elastic/docs.