Keep order of operands in AndTypes invariant

We previously could get the following:

Given:

    class A
    trait B extends A
    trait C extends A

Then

    (A & B) & C

simplified to

    C & B

This is usually not a problem, *except* for typing super calls, where we pick
the symbol of the right operand. The problem was masked because due to the wrong
computation of base classes for AndTypes, we always ended up picking the leftmost
symbol in a joint denotation instead of the rightmost one. So we got a double swap!

The problem is reproducible in the test case. It originally manifested itself only
in the bootstrap, and only under a particular order of compiling files, which was
exercised by the CI but initially not by my local setup. The summary of the problem is as follows:

ElimRepeated has the structure of ER in the test case. Because of
the problem, a super call in `transform` of `ElimRepeated` went to `InfoTransformers` instead
of `AnnotationTransformers`. Consequently, the elim repeated transform was not done on
annotations, which led to backend crashes of the bootstrapped compiler for those annotations
that had repeated arguments. It took me two full days to get to the bottom of this.
Must rank right at the top of the list of hard-to-find bugs for me.

Note: It would seem logical to apply a similar technique to OrTypes. But this currently fails some tests. Need to investigate later.

Author: odersky

compiler/src/dotty/tools/dotc/core/TypeComparer.scala

@@ -1143,11 +1143,11 @@ class TypeComparer(initctx: Context) extends DotClass with ConstraintHandling {
           case OrType(tp11, tp12) =>
             tp11 & tp2 | tp12 & tp2
           case _ =>
-            val t1 = mergeIfSub(tp1, tp2)
-            if (t1.exists) t1
+            val tp1a = dropIfSuper(tp1, tp2)
+            if (tp1a ne tp1) glb(tp1a, tp2)
             else {
-              val t2 = mergeIfSub(tp2, tp1)
-              if (t2.exists) t2
+              val tp2a = dropIfSuper(tp2, tp1)
+              if (tp2a ne tp2) glb(tp1, tp2a)
               else tp1 match {
                 case tp1: ConstantType =>
                   tp2 match {
@@ -1161,7 +1161,7 @@ class TypeComparer(initctx: Context) extends DotClass with ConstraintHandling {
                       // However, maybe we can still apply the replacement to
                       // types which are not explicitly written.
                       defn.NothingType
-                    case _ => andType(tp1, tp2)
+                   case _ => andType(tp1, tp2)
                   }
                 case _ => andType(tp1, tp2)
               }
@@ -1204,6 +1204,22 @@ class TypeComparer(initctx: Context) extends DotClass with ConstraintHandling {
   final def lub(tps: List[Type]): Type =
     ((defn.NothingType: Type) /: tps)(lub(_,_, canConstrain = false))
 
+  private def recombineAndOr(tp: AndOrType, tp1: Type, tp2: Type) =
+    if (!tp1.exists) tp2
+    else if (!tp2.exists) tp1
+    else tp.derivedAndOrType(tp1, tp2)
+
+  /** If some (&-operand of) this type is a supertype of `sub` replace it with `NoType`.
+   */
+  private def dropIfSuper(tp: Type, sub: Type): Type =
+    if (isSubTypeWhenFrozen(sub, tp)) NoType
+    else tp match {
+      case tp @ AndType(tp1, tp2) =>
+        recombineAndOr(tp, dropIfSuper(tp1, sub), dropIfSuper(tp2, sub))
+      case _ =>
+        tp
+    }
+
   /** Merge `t1` into `tp2` if t1 is a subtype of some &-summand of tp2.
    */
   private def mergeIfSub(tp1: Type, tp2: Type): Type =

tests/run/supercalls-traits.check

@@ -0,0 +1,5 @@
+C1A1A2B1B2C2
+C1B3B4C3
+IT
+AT
+ER

tests/run/supercalls-traits.scala

@@ -14,9 +14,42 @@ class Base[A](exp: => Option[A])
 
 object Empty extends Base[Nothing](None)
 
+
+trait B1 extends C1 { override def f() = { super.f(); print("B1") }}
+trait B2 extends B1 { override def f() = { super.f(); print("B2") }}
+trait A1 extends C1 { override def f() = { super.f(); print("A1") }}
+trait A2 extends A1 { override def f() = { super.f(); print("A2") }}
+class C1 { def f() = print("C1") }
+class C2 extends A2 with B2 { override def f() = { super.f(); print("C2") }}
+
+
+trait B3 extends C1 { override def f() = { super.f(); print("B3") }}
+trait B4 extends C1 { this: B3 =>  override def f() = { super.f(); print("B4") }}
+class C3 extends C1 with B3 with B4 { override def f() = { super.f(); print("C3") }}
+
+trait DT {
+  def f(): Unit
+}
+trait IT extends DT {
+  def f() = { println("IT") }
+}
+abstract class MPT {
+}
+trait AT extends MPT with DT {
+  abstract override def f() = { super.f(); println("AT") }
+}
+class ER extends MPT with IT with AT {
+  override def f() = { super.f(); println("ER") }
+}
+
 object Test {
   def main(args: Array[String]): Unit = {
     assert(new C().foo == 3)
+    new C2().f()
+    println()
+    new C3().f()
+    println()
+    new ER().f()
     Empty
   }
 }