Track type variable dependencies to guide instantiation decisions (#16042)

We now keep track of reverse type variable dependencies in constraints. 
E.g. if a constraint contains a clause like

    A >: List[B]

We associate with `B` info that `A` depends co-variantly on it. Or, if 

    A <: B => C

we associate with `B` that `A` depends co-variantly on it and with `C`
that `A` depends contra-variantly on it. Here, co-variant means that
the allowable range of `A` narrows if the referred-to variable `B`
grows, and
contra-variant means that the allowable range of `A` narrows if the
referred-to
variable `C` shrinks. If there's an invariant reference such as

    A <: Array[B]

Then `A` depends both co- and contra-variantly on `B`.

These dependencies are then used to guide type variable instantiation. 
If an eligible type variable does not appear in the type of a typed
expression,
we interpolate it to one of its bounds. Previously this was done in an
ad-hoc
manner where we minimized the type variable if it had a lower bound and
maximized
it otherwise. We now take reverse dependencies into account. If
maximizing a type
variable would narrow the remaining constraint we minimize, and if
minimizing
a type variable would narrow the remaining constraint we maximize. Only
if
the type variable is not referred to from the remaining constraint we
resort
to the old heuristic based on the lower bound. 

Fixes #15864

Todo: This could be generalized in several directions:

- We could base the dependency tracking on type param refs instead of
type variables.
That could make the `replace` operation in a constraint more efficient.
- We could base more interpolation decisions on dependencies. E.g. we
could
interpolate a type variable only if both the type of an expression and
the enclosing
   constraint agree in which direction this should be done.

Author: odersky

compiler/src/dotty/tools/dotc/config/Config.scala

@@ -22,6 +22,11 @@ object Config {
    */
   inline val checkConstraintsNonCyclic = false
 
+  /** Check that reverse dependencies in constraints are correct and complete.
+   *  Can also be enabled using -Ycheck-constraint-deps.
+   */
+  inline val checkConstraintDeps = false
+
   /** Check that each constraint resulting from a subtype test
    *  is satisfiable. Also check that a type variable instantiation
    *  satisfies its constraints.
@@ -184,6 +189,9 @@ object Config {
   /** If set, prints a trace of all symbol completions */
   inline val showCompletions = false
 
+  /** If set, show variable/variable reverse dependencies when printing constraints. */
+  inline val showConstraintDeps = true
+
   /** If set, method results that are context functions are flattened by adding
    *  the parameters of the context function results to the methods themselves.
    *  This is an optimization that reduces closure allocations.

compiler/src/dotty/tools/dotc/config/ScalaSettings.scala

@@ -309,6 +309,7 @@ private sealed trait YSettings:
   val YforceSbtPhases: Setting[Boolean] = BooleanSetting("-Yforce-sbt-phases", "Run the phases used by sbt for incremental compilation (ExtractDependencies and ExtractAPI) even if the compiler is ran outside of sbt, for debugging.")
   val YdumpSbtInc: Setting[Boolean] = BooleanSetting("-Ydump-sbt-inc", "For every compiled foo.scala, output the API representation and dependencies used for sbt incremental compilation in foo.inc, implies -Yforce-sbt-phases.")
   val YcheckAllPatmat: Setting[Boolean] = BooleanSetting("-Ycheck-all-patmat", "Check exhaustivity and redundancy of all pattern matching (used for testing the algorithm).")
+  val YcheckConstraintDeps: Setting[Boolean] = BooleanSetting("-Ycheck-constraint-deps", "Check dependency tracking in constraints (used for testing the algorithm).")
   val YretainTrees: Setting[Boolean] = BooleanSetting("-Yretain-trees", "Retain trees for top-level classes, accessible from ClassSymbol#tree")
   val YshowTreeIds: Setting[Boolean] = BooleanSetting("-Yshow-tree-ids", "Uniquely tag all tree nodes in debugging output.")
   val YfromTastyIgnoreList: Setting[List[String]] = MultiStringSetting("-Yfrom-tasty-ignore-list", "file", "List of `tasty` files in jar files that will not be loaded when using -from-tasty")

compiler/src/dotty/tools/dotc/core/Constraint.scala

@@ -4,6 +4,7 @@ package core
 
 import Types._, Contexts._
 import printing.Showable
+import util.{SimpleIdentitySet, SimpleIdentityMap}
 
 /** Constraint over undetermined type parameters. Constraints are built
  *  over values of the following types:
@@ -128,7 +129,7 @@ abstract class Constraint extends Showable {
 
   /** Is `tv` marked as hard in the constraint? */
   def isHard(tv: TypeVar): Boolean
-  
+
   /** The same as this constraint, but with `tv` marked as hard. */
   def withHard(tv: TypeVar)(using Context): This
 
@@ -165,15 +166,32 @@ abstract class Constraint extends Showable {
    */
   def hasConflictingTypeVarsFor(tl: TypeLambda, that: Constraint): Boolean
 
-  /** Check that no constrained parameter contains itself as a bound */
-  def checkNonCyclic()(using Context): this.type
-
   /** Does `param` occur at the toplevel in `tp` ?
    *  Toplevel means: the type itself or a factor in some
    *  combination of `&` or `|` types.
    */
   def occursAtToplevel(param: TypeParamRef, tp: Type)(using Context): Boolean
 
+  /** A string that shows the reverse dependencies maintained by this constraint
+   *  (coDeps and contraDeps for OrderingConstraints).
+   */
+  def depsToString(using Context): String
+
+  /** Does the constraint restricted to variables outside `except` depend on `tv`
+   *  in the given direction `co`?
+   *  @param `co`  If true, test whether the constraint would change if the variable is made larger
+   *               otherwise, test whether the constraint would change if the variable is made smaller.
+   */
+  def dependsOn(tv: TypeVar, except: TypeVars, co: Boolean)(using Context): Boolean
+
+  /** Depending on Config settngs:
+   *   - Under `checkConstraintsNonCyclic`, check that no constrained
+   *     parameter contains itself as a bound.
+   *   - Under `checkConstraintDeps`, check hat reverse dependencies in
+   *     constraints are correct and complete.
+   */
+  def checkWellFormed()(using Context): this.type
+
   /** Check that constraint only refers to TypeParamRefs bound by itself */
   def checkClosed()(using Context): Unit