GH-1911 - Provide a way for defining dynamic queries in a type safe way.

This commit adds a couple of additional Spring Data Neo4j Extensions (mixins):

* `QuerydslPredicateExecutor`
* `CypherdslConditionExecutor`
* `CypherdslStatementExecutor`
* `ReactiveCypherdslStatementExecutor`

SDN provides now two different ways of adding dynamic conditions to generated queries: By supporting the existing Spring Data Commons `QuerydslPredicateExecutor` via the new support of QueryDSL expressions inside Cypher-DSL 2021.1.1 as well as the Neo4j native pendant to it, the `CypherdslConditionExecutor`. Conceptionally, both mixins provide the same feature. Both are implemented via an additional repository fragment, that is linked to the repository via the `Neo4jRepositoryFactory`.

The Cypher-DSL statement executors come as a pair: For the imperative and reactive world. The interface provides overloads for `findOne` and `finaAll`, which all take at least a Cypher-DSL statement, so that these extensions can be used to specifiy a complete query in a type safe way. Also provided are overloads taking in a type parameter that allows to specify a project, thus creating symmetry with the standard repository and their string based finder methods with a type parameter.

Implementationwise both statement executors are provided via new variants of a `AbstractNeo4jQuery` respectively `AbstractReactiveNeo4jQuery`, as we didn’t find a solution to access the projection type from a fragment.

This closes #1911.

Author: michael-simons

pom.xml

@@ -359,7 +359,12 @@
 			<artifactId>neo4j</artifactId>
 			<scope>test</scope>
 		</dependency>
-
+		<dependency>
+			<groupId>com.querydsl</groupId>
+			<artifactId>querydsl-core</artifactId>
+			<version>${querydsl}</version>
+			<scope>provided</scope>
+		</dependency>
 	</dependencies>
 
 	<repositories>

src/main/asciidoc/faq/faq.adoc

@@ -204,7 +204,7 @@ public interface MyPersonRepository extends Neo4jRepository<Person, Long> {
     Also, the `Pageable` should be unsorted and you should provide a stable order.
     We won't use the sorting information from the pageable.
 <.> This method returns a page. A page knows about the exact number of total pages.
-    Therefore you must specify an additional count query.
+    Therefore, you must specify an additional count query.
     All other restrictions from the second method apply.
 
 [[faq.custom-queries-and-custom-mappings]]
@@ -219,15 +219,31 @@ object to write the data back which will eventually erase or overwrite data you
 So, please use repositories and declarative methods with `@Query` in all cases where the result is shaped like your domain
 model or you are sure you don't use a partially mapped model for write commands.
 
-What are the alternatives? First, please read up on two things: <<repositories.custom-implementations,custom repository fragments>>
+What are the alternatives?
+
+* <<projections.sdn.general-remarks,Projections>> might be already enough to shape your *view* on the graph: They can be used to define
+  the depth of fetching properties and related entities in an explicit way: By modelling them.
+* If your goal is to make only the conditions of your queries *dynamic*, then have a look at the <<core.extensions.querydsl, `QuerydslPredicateExecutor`>>
+  but especially our own variant of it, the `CypherdslConditionExecutor`. Both <<sdn-mixins,mixins>> allow to add conditions to
+  the full queries we create for you. Thus, you will have the domain fully populated together with custom conditions.
+  Of course, your conditions must work with what we generate. Find the names of the root node, the related nodes and more
+  <<custom-queries,here>>.
+* Use the http://neo4j-contrib.github.io/cypher-dsl/current/[Cypher-DSL] via the `CypherdslStatementExecutor` or the `ReactiveCypherdslStatementExecutor`.
+  The Cypher-DSL is predestined to create dynamic queries. In the end, it's what SDN uses under the hood anyway. The corresponding
+  mixins work both with the domain type of a repository itself as well as with projections (something that the mixins for adding
+  conditions don't).
+
+If you think that you can solve your problem with a partially dynamic query or a full dynamic query together with a projection,
+please jump back now to the chapter <<sdn-mixins, about Spring Data Neo4j Mixins>>.
+
+Otherwise, please read up on two things: <<repositories.custom-implementations,custom repository fragments>>
 the <<sdn-building-blocks,levels of abstractions>> we offer in SDN 6.
 
-Why speaking about custom repository fragments?
+Why speaking about custom repository fragments now?
 
-* You might want to use the Cypher-DSL for building type-safe queries
-* You might have more complex situation in which a dynamic query is required, but the query still belongs
+* You might have more complex situation in which more than one dynamic query is required, but the queries still belong
   conceptually in a repository and not in the service layer
-* Your custom query returns a graph shaped result that fits not quite to your domain model
+* Your custom queries return a graph shaped result that fits not quite to your domain model
   and therefore the custom query should be accomponied by a custom mapping as well
 * You have the need for interacting with the driver, i.e. for bulk loads that should not go through object mapping.
 

src/main/asciidoc/getting-started/getting-started.adoc

@@ -133,7 +133,7 @@ Our domain layer should accomplish two things:
 === Example Node-Entity
 
 SDN fully supports unmodifiable entities, for both Java and `data` classes in Kotlin.
-Therefore we will focus on immutable entities here, <<movie-entity>> shows a such an entity.
+Therefore, we will focus on immutable entities here, <<movie-entity>> shows a such an entity.
 
 NOTE: SDN supports all data types the Neo4j Java Driver supports, see https://neo4j.com/docs/driver-manual/current/cypher-workflow/#driver-type-mapping[Map Neo4j types to native language types] inside the chapter "The Cypher type system".
 Future versions will support additional converters.
@@ -229,7 +229,7 @@ Most of the time it's harder to take something away, than to add stuff afterward
 Furthermore, using store specifics leaks your store into your domain.
 From a performance point of view, there is no penalty.
 
-A repository fitting to any of the movie entities above looks like this:
+A reactive repository fitting to any of the movie entities above looks like this:
 
 [source,java]
 [[movie-repository]]

src/main/asciidoc/index.adoc

@@ -33,7 +33,6 @@ NOTE: Copies of this document may be made for your own use and for distribution
 
 include::introduction-and-preface/index.adoc[]
 include::{spring-data-commons-docs}/dependencies.adoc[leveloffset=+1]
-include::{spring-data-commons-docs}/repositories.adoc[leveloffset=+1]
 
 [[reference]]
 = Reference Documentation
@@ -49,6 +48,12 @@ include::getting-started/index.adoc[]
 
 include::object-mapping/index.adoc[]
 
+include::{spring-data-commons-docs}/repositories.adoc[leveloffset=+1]
+
+include::{spring-data-commons-docs}/query-by-example.adoc[leveloffset=+2]
+
+include::object-mapping/sdn-extensions.adoc[leveloffset=+1]
+
 include::{spring-data-commons-docs}/repository-projections.adoc[leveloffset=+1]
 
 include::object-mapping/projections.adoc[leveloffset=+1,lines=4..]

src/main/asciidoc/object-mapping/projections.adoc

@@ -126,12 +126,23 @@ interface TestRepository extends CrudRepository<TestEntity, Long> { // <.>
 <.> Methods returning one or more `TestEntity` will just return instances of it, as it matches the domain type
 <.> Methods returning one or more instances of class that extend the domain type will just return instances
     of the extending class. The domain type of the method in question will the extended class, which
-    still satifies the domain type of the repository itself
+    still satisfies the domain type of the repository itself
 <.> This method returns an interface projection, the return type of the method is therefore different
     from the repository's domain type. The interface can only access properties defined in the domain type
 <.> This method returns a DTO projection. Executing it will cause SDN to issue a warning, as the DTO defines
     `numberOfRelations` as additional attribute, which is not in the contract of the domain type.
     The annotated attribute `aProperty` in `TestEntity` will be correctly translated to `a_property` in the query.
-    As above, the return type is different from the repositories domain type.
+    As above, the return type is different from the repositories' domain type.
 <.> This method also returns a DTO projection. However, no warning will be issued, as the query contains a fitting
     value for the additional attributes defined in the projection.
+
+TIP: While the repository in <<projections.simple-entity-repository,the listing above>> uses a concrete return type to
+     define the projection, another variant is the use of <<projection.dynamic,dynamic projections>> as explained in the
+     parts of the documentation Spring Data Neo4j shares with other Spring Data Projects. A dynamic projection can be
+     applied to both closed and open interface projections as well as to class based DTO projections:
+     +
+     +
+     The key to a dynamic projection is to specifiy the desired projection type as the last parameter to a query method
+     in a repository like this: `<T> Collection<T> findByName(String name, Class<T> type)`. This is a declaration that
+     could be added to the `TestRepository` above and allow for different projections retrieved by the same method, without
+     to repeat a possible `@Query` annotation on several methods.

src/main/asciidoc/object-mapping/sdn-extensions.adoc

@@ -0,0 +1,98 @@
+[[sdn-mixins]]
+== Spring Data Neo4j Extensions
+
+=== Available extensions for Spring Data Neo4j repositories
+
+Spring Data Neo4j offers a couple of extensions or "mixins" that can be added to repositories. What is a mixin? According to
+https://en.wikipedia.org/wiki/Mixin[Wikipedia] mixins are a language concept that allows a programmer to inject some code
+into a class. Mixin programming is a style of software development, in which units of functionality are created in a class
+and then mixed in with other classes.
+
+Java does not support that concept on the language level, but we do emulate it via a couple of interfaces and a runtime
+that adds appropriate implementations and interceptors for.
+
+Mixins added by default are `QueryByExampleExecutor` and `ReactiveQueryByExampleExecutor` respectively. Those interfaces are
+explained in detail in <<query-by-example>>.
+
+Additional mixins provided are:
+
+* `QuerydslPredicateExecutor`
+* `CypherdslConditionExecutor`
+* `CypherdslStatementExecutor`
+* `ReactiveCypherdslStatementExecutor`
+
+[[sdn-mixins.dynamic-conditions]]
+==== Add dynamic conditions to generated queries
+
+Both the `QuerydslPredicateExecutor` and `CypherdslConditionExecutor` provide the same concept: SDN generates a query, you
+provide "predicates" (Query DSL) or "conditions" (Cypher DSL) that will be added. We recommend the Cypher DSL, as this is
+what SDN 6 uses natively. You might even want to consider using the
+http://neo4j-contrib.github.io/cypher-dsl/2021.1.1/#thespringdataneo4j6annotationprocessor[annotation processor] that generates
+a static meta model for you.
+
+How does that work? Declare your repository as described above and add *one*  of the following interfaces:
+
+[source,java,indent=0,tabsize=4]
+----
+include::../../../../src/test/java/org/springframework/data/neo4j/integration/imperative/QuerydslNeo4jPredicateExecutorIT.java[tags=sdn-mixins.dynamic-conditions.add-mixin]
+----
+<.> Standard repository declaration
+<.> The Query DSL mixin
+
+*OR*
+
+[source,java,indent=0,tabsize=4]
+----
+include::../../../../src/test/java/org/springframework/data/neo4j/integration/imperative/CypherdslConditionExecutorIT.java[tags=sdn-mixins.dynamic-conditions.add-mixin]
+----
+<.> Standard repository declaration
+<.> The Cypher DSL mixin
+
+Exemplary usage is shown with the Cypher DSL condition executor:
+
+[source,java,indent=0,tabsize=4]
+----
+include::../../../../src/test/java/org/springframework/data/neo4j/integration/imperative/CypherdslConditionExecutorIT.java[tags=sdn-mixins.dynamic-conditions.usage]
+----
+<.> Define a named `Node` object, targeting the root of the query
+<.> Derive some properties from it
+<.> Create an `or` condition. An anonymous parameter is used for the first name, a named parameter for
+the last name. This is how you define parameters in those fragments and one of the advantages over the Query-DSL
+mixin which can't do that.
+Literals can be expressed with `Cypher.literalOf`.
+<.> Define a `SortItem` from one of the properties
+
+The code looks pretty similar for the Query-DSL mixin. Reasons for the Query-DSL mixin can be familiarity of the API and
+that it works with other stores, too. Reasons against it are the fact that you need an additional library on the class path,
+it's missing support for traversing relationships and the above mentioned fact that it doesn't support parameters in its
+predicates (it technically does, but there are no API methods to actually pass them to the query being executed).
+
+[[sdn-mixins.using-cypher-dsl-statements]]
+==== Using (dynamic) Cypher-DSL statements for entities and projections
+
+Adding the corresponding mixin is not different than using the <<sdn-mixins.dynamic-conditions, condition excecutor>>:
+
+[source,java,indent=0,tabsize=4]
+----
+include::../../../../src/test/java/org/springframework/data/neo4j/integration/imperative/CypherdslStatementExecutorIT.java[tags=sdn-mixins.using-cypher-dsl-statements.add-mixin]
+----
+
+Please use the `ReactiveCypherdslStatementExecutor` when extending the `ReactiveNeo4jRepository`.
+
+The `CypherdslStatementExecutor` comes with several overloads for `findOne` and `findAll`. They all take a Cypher-DSL
+statement respectively an ongoing definition of that as a first parameter and in case of the projecting methods, a type.
+
+If a query requires parameters, they must be defined via the Cypher-DSL itself and also populated by it, as the following listing shows:
+
+[source,java,indent=0,tabsize=4]
+----
+include::../../../../src/test/java/org/springframework/data/neo4j/integration/imperative/CypherdslStatementExecutorIT.java[tags=sdn-mixins.using-cypher-dsl-statements.using]
+----
+<.> The dynamic query is build in a type safe way in a helper method
+<.> We already saw this in <<sdn-mixins.dynamic-conditions,here>>, where we also defined some variables holding the model
+<.> We define an anonymous parameter, filled by the actual value of `name`, which was passed to the method
+<.> The statement returned from the helper method is used to find an entity
+<.> Or a projection.
+
+The `findAll` methods works similar.
+The imperative Cypher-DSL statement executor also provides an overload returning paged results.

src/main/java/org/springframework/data/neo4j/core/Neo4jTemplate.java

@@ -676,29 +676,29 @@ public T getRequiredSingleResult() {
 		private Optional<Neo4jClient.RecordFetchSpec<T>> createFetchSpec() {
 			QueryFragmentsAndParameters queryFragmentsAndParameters = preparedQuery.getQueryFragmentsAndParameters();
 			String cypherQuery = queryFragmentsAndParameters.getCypherQuery();
-			Map<String, Object> finalParameters = preparedQuery.getQueryFragmentsAndParameters().getParameters();
+			Map<String, Object> finalParameters = queryFragmentsAndParameters.getParameters();
 
 			QueryFragmentsAndParameters.QueryFragments queryFragments = queryFragmentsAndParameters.getQueryFragments();
 			Neo4jPersistentEntity<?> entityMetaData = (Neo4jPersistentEntity<?>) queryFragmentsAndParameters.getNodeDescription();
 
 			boolean containsPossibleCircles = entityMetaData != null && entityMetaData.containsPossibleCircles(queryFragments::includeField);
 			if (cypherQuery == null || containsPossibleCircles) {
 
-				Map<String, Object> parameters = queryFragmentsAndParameters.getParameters();
-
 				if (containsPossibleCircles && !queryFragments.isScalarValueReturn()) {
 					GenericQueryAndParameters genericQueryAndParameters =
-							createQueryAndParameters(entityMetaData, queryFragments, parameters);
+							createQueryAndParameters(entityMetaData, queryFragments, queryFragmentsAndParameters.getParameters());
 
 					if (genericQueryAndParameters.isEmpty()) {
 						return Optional.empty();
 					}
 					cypherQuery = renderer.render(queryFragments.generateGenericStatement());
 					finalParameters = genericQueryAndParameters.getParameters();
 				} else {
-					cypherQuery = renderer.render(queryFragments.toStatement());
+					Statement statement = queryFragments.toStatement();
+					cypherQuery = renderer.render(statement);
+					finalParameters = new HashMap<>(finalParameters);
+					finalParameters.putAll(statement.getParameters());
 				}
-
 			}
 
 			Neo4jClient.MappingSpec<T> newMappingSpec = neo4jClient.query(cypherQuery)
@@ -715,9 +715,11 @@ private GenericQueryAndParameters createQueryAndParameters(Neo4jPersistentEntity
 					.prepareMatchOf(entityMetaData, queryFragments.getMatchOn(), queryFragments.getCondition())
 					.returning(Constants.NAME_OF_SYNTHESIZED_ROOT_NODE).build();
 
+			Map<String, Object> usedParameters = new HashMap<>(parameters);
+			usedParameters.putAll(rootNodesStatement.getParameters());
 			final Collection<Long> rootNodeIds = new HashSet<>((Collection<Long>) neo4jClient
 					.query(renderer.render(rootNodesStatement))
-					.bindAll(parameters)
+					.bindAll(usedParameters)
 					.fetch()
 					.one()
 					.map(values -> values.get(Constants.NAME_OF_SYNTHESIZED_ROOT_NODE))
@@ -737,6 +739,8 @@ private GenericQueryAndParameters createQueryAndParameters(Neo4jPersistentEntity
 						.prepareMatchOf(entityMetaData, relationshipDescription, queryFragments.getMatchOn(), queryFragments.getCondition())
 						.returning(cypherGenerator.createReturnStatementForMatch(entityMetaData)).build();
 
+				usedParameters = new HashMap<>(parameters);
+				usedParameters.putAll(statement.getParameters());
 				neo4jClient.query(renderer.render(statement))
 						.bindAll(parameters)
 						.fetch()
@@ -791,5 +795,4 @@ private Consumer<Map<String, Object>> iterateAndMapNextLevel(Set<Long> relations
 			};
 		}
 	}
-
 }