My original plan was that, upon returning from my vacation, I would spend my remaining time at EPFL writing a technical report explaining everything I knew about the problems with Scala Classic. Instead, on my first day back in the office, Martin came by with a draft of a new formal system, DOT (for Dependent Object Types), that he came up with while he was on vacation. After about four weeks I managed to root out pretty much all of the obvious problems with DOT, but another four weeks was not enough to get the proofs and the metatheory into a state that we were happy with. I am not really sure what will happen with DOT in the long term.

There are a few clever things about DOT that avoid some of the problems I encountered with Scala Classic. For example, Scala Classic only had intersection types while DOT has both intersection and union types, which solves some problems with computing the members of types.  However, with Scala Classic I was trying to model a minimal subset Scala language as it exists, without adding any new features that many never be supported by the full language. I have no idea if we will see union types in the actual Scala implementation anytime soon.

The other thing DOT does is give up the goal of trying to be a minimal subset of Scala and throws out quite a few important things from a user perspective. For example, there are no methods (only λ-abstractions), no existentials, no built-in inheritance or mix-in composition (though you can encode it by hand), and object fields must be syntactically values.

This last restriction is particularly important because it solves by fiat the problems that arose in Scala Classic from using paths that have an empty type in dependent type projections. However, it also means that you may not be able to directly encode some Scala programs into DOT without an effect or termination analysis. Therefore, while DOT has the potential to be a useful intermediary step, there will still be more work to be done to provide a semantics for a more realistic subset of the Scala language.

I have been in Copenhagen for a little over a month now, but I am not really ready to comment on the research I have been doing here yet.  So in the meantime I'll probably focus more on fonts and typography.

Overall it was far less work than I was expecting.  Scala already internally supports what it calls "constant types", there is just no way to write them in Scala source code presently.  Consequently, most of the effort was in extending the parser.

Given my modifications, it is now possible to write code like the following:

 
scala> val x : "foo".type = "foo"
x: "foo".type = foo
 

What I was not expecting was that out-of-the-box things like the following would work:

 
scala> val x : "foobar".type = "foo" + "bar"
x: "foobar".type = foobar
scala> val y : 10.type = 2 * 5
y: 10.type = 10
scala> def frob(arg : 10.type) : 6.type = arg - 4
frob: (10.type)6.type
 

Unfortunately the excitement soon passes when you realize all the things you can't do with singleton literals (yet). Even if we turn on the experimental dependent method support, you can't write things like

 
def add(arg : Int) : (arg + 5).type = arg + 5
 

because these are exactly what they are called, singleton literals, not full-blown dependent types.

One cute example, based on a use suggested by Sean McDirmid, would be that some people might do something like the following with implicits:

 
implicit stringToColor(arg : String) : java.awt.Color = java.awt.Color.getColor(arg)
 

However, with singleton literals you can tighten it up by ensuring that for some strings it will never fail:

 
implicit redToColor(arg : "red".type) : java.awt.Color = java.awt.Color.RED
implicit blueToColor(arg : "blue".type) : java.awt.Color = java.awt.Color.BLUE
 

Happily, the type inferencer already chooses the most specific implicit conversions.

In any event, they will hopefully become more useful as the type system continues to grow. I am also sure someone will probably come up with a clever use for them that hasn't occurred to me yet. If so, let me know.

Several weeks ago I decided that instead of calling the core calculus that I have been working on Featherweight Scala, which could be confusing given that there is already a different calculus with that name, that I would call it Scala Classic.

I had hoped to submit a paper on Scala Classic to POPL 2009, but it has been taking too long to work out the metatheory. Partly because I am developing a mechanized proof of type soundness in Twelf, and on my first attempt at representing heaps I think tried to be too clever.  However, a more traditional treatment of heaps leads to the need to explicitly represent typing contexts, rather than the implicit treatment of contexts more commonly used in Twelf.

This afternoon I finally finished the proof of the theorem that is traditionally called "substitution".  However, the proof of type preservation will also require another theorem formalizing a substitution-like property for singleton types.  I am hoping that I can prove that theorem more quickly now that I've built up all the explicit context theory.  I have not thought much about whether there are any unusual theorems needed to prove progress.

In any event, I would ideally have the proofs finished by the beginning of August.  My current plan is to try submitting the paper to PLDI (which is even in Europe this year), unless some obviously better venue comes to my attention.

One of the reasons I would really like to have the proofs done by August is that I will be teaching a PhD level course here at EPFL this fall on mechanized sepcifications and proofs, with a strong emphasis on deductive systems. It also looks like I may fill in some for a masters level course based on TAPL.  So between those two things, I am not going to have nearly as much time for research as would be ideal.

So here is my example of how the encoding works. First, the SML version:

 
signature Eq = sig
  type t
  val eq: t -> t -> bool
end
 
signature RicherEq = sig
  include Eq
  val neq: t -> t -> bool
end
 
functor mkRicherEq(arg : Eq) :> RicherEq where type t = arg.t = struct
  type t = arg.t
  val eq = arg.eq
  fun neq x y = not (eq x y)
end
 

We can transliterate this example into Scala as:

 
type Eq = {
  type T
  def eq(x: T, y: T): Boolean
}
 
type RicherEq = {
  type T
  def eq(x: T, y: T): Boolean
  def neq(x: T, y: T): Boolean
}
 
def mkRicherEq(arg: Eq) : RicherEq { type T = arg.T } = new {
  type T = arg.T
  def eq(x: T, y: T) = arg.eq(x, y)
  def neq(x: T, y:T) = !eq(x, y)
}
 

The only problem I discovered is that it is not possible to define RicherEq in terms of Eq as we could in SML:

 
scala> type RicherEq = Eq { def neq(x: T, y: T): Boolean }
<console>:5: error: Parameter type in structural refinement may
not refer to abstract type defined outside that same refinement
       type RicherEq = Eq { def neq(x: T, y: T): Boolean }
                                       ^
<console>:5: error: Parameter type in structural refinement may
not refer to abstract type defined outside that same refinement
       type RicherEq = Eq { def neq(x: T, y: T): Boolean }
                                             ^
 

Why this restriction exists I don't know. In fact, this sort of refinement should work in the current version of Featherweight Scala, so perhaps it can be lifted eventually.

I still need to think about higher-order functors, and probably spend a few minutes researching existing proposals. I think this is probably something that cannot be easily supported in Scala if it will require allowing method invocations to appear in paths. However, off hand that only seems like it should be necessary for applicative higher-order functors, but again I definitely need to think it through.

For example, consider the following SML signature:

 
signature Nat = sig
  type t
  val z: t
  val s: t -> t
end
 

This signature can be translated in to Scala as:

 
type Nat = {
  type T
  val z: T
  def s(arg: T): T
}
 

It is then possible to create an implementation of this type, and opaquely seal it (hiding the definition of T). In SML:

 
structure nat :> Nat = struct
  type t = int
  val z = 0
  fun s n = n + 1
end
 

In Scala:

 
val nat : Nat = new {
  type T = Int
  val z = 0
  def s(arg: Int) = arg + 1
}
 

In many cases when programming with SML modules it is necessary or convenient to give a module that reveals the definition of an abstract type. In the above example, this can be done by adding a where type clause to the first line:

 
structure nat :> Nat where type t = int = struct
...
 

We can do the same thing in Scala using refinements:

 
val nat : Nat { type T = Int } = new {
...
 

Great, right? Well, almost. The problem is that structural types are still a bit buggy in Scala compiler at present. So, while the above typechecks, you can't quite use it yet:

 
scala> nat.s(nat.z)
java.lang.NoSuchMethodException: $anon$1.s(java.lang.Object)
        at java.lang.Class.getMethod(Class.java:1581)
        at .reflMethod$Method1(<console>:7)
        at .<init>(<console>:7)
        at .<clinit>(<console>)
        at RequestResult$.<init>(<console>:3)
        at RequestResult$.<clinit>(<console>)
        at RequestResult$result(<console>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflec...
 

There were some issues raised about how faithful an encoding of SML functors, and well-known extensions for higher-order functors, one can get in Scala. Indeed, off the top of my head it is not entirely clear. So I need to think more about that before I write some examples.

Probably what makes working out the design of Featherweight Scala so difficult is that I am trying my best to commit to it being a valid subset of Scala. It seems like whenever I try to simplify one part of the language, it just introduces a complication elsewhere. Still, I think so far that I have been able to ensure that it is a subset with two exceptions.

The first exception is that some programs in Featherweight Scala will diverge in cases where the Scala compiler will generate a terminating program. This is because in programs that attempt to access an uninitialized field, Scala will return null, while Featherweight Scala diverges. Since the program many never attempt to use the null value, it may simple terminate without raising a NullPointerException.

The second exception arises because type checking in Featherweight Scala's type system is most likely undecidable. Given that the design is in flux, I haven't precisely verified this, but it seems quite plausible given its expressive power is similar to languages where type checking is known to be undecidable. Because type checking is undecidable, and the Scala language implementation attempts to have its type checking phase be terminating, there should be some programs that can be shown to be well-typed in Featherweight Scala that the Scala compiler will reject.

I think the other thing I've determined is that I do not understand F-bounded quantification as well as I thought I did. At least, I find myself worried about the cases where the typing rules presently require  templates, that have not yet been verified to be well-formed, to be introduced as hypotheses while verifying their own correctness. So I need to do some additional reading to see how some other languages deal with this issue. Certainly I can put my mind at ease by making the type system more restrictive, but I do not want to be so restrictive that it is no longer reasonable to call the language Featherweight Scala.

Update: Okay, so after looking at Featherweight Generic Java and νObj again more closely, they both do essentially the same kind of thing. In FGJ, the class table is well-formed if all its classes are well-formed, but checking a class's well-formedness may require presupposing that that class tale is well-formed. In νObj, the well-formedness of recursive record types is checked under the assumption that the record's self-name has the type of the record. Certainly the same sort of things come up when verifying the well-formedness of recursive functions and recursive types, but at least there what is being assumed is at a higher level of abstraction: when checking a recursive function you assume that the function has some well-formed type, or when checking a recursive type you assume that the type has some well-formed kind.

First we can define an infinite family of traits to represent n-ary universal quantification, much like Scala represents n-ary functions types:

 
trait Univ1[Bound1,Body[_]] {
   def Apply[T1<:Bound1] : Body[T1]
}
 
trait Univ2[Bound1,Bound2,Body[_,_]] {
   def Apply[T1<:Bound1,T2<:Bound2] : Body[T1,T2]
}
 
// ... so on for N > 2
 

Really, the key to this encoding is the use of higher-kinded quantification to encode the binding structure of the quantifier.

Now it is possible to write some examples similar to what I gave previously, but more concisely:

 
object Test extends Application {
   def id[T](x : T) = x
 
   type Id[T] = T => T
   val id = new Univ1[Any,Id]
     { def Apply[T <: Any] : Id[T] = id[T] _ }
 
   val idString = id.Apply[String]
   val idStringList = id.Apply[List[String]]
 
   println(idString("Foo"))
   println(idStringList(List("Foo", "Bar", "Baz")))
 
   type Double[T] = T => (T, T)
   val double = new Univ1[Any,Double]
     { def Apply[T <: Any] : Double[T] = (x : T) => (x, x) }
 
   val doubleString = double.Apply[String]
   val doubleStringList = double.Apply[List[String]]
 
   println(doubleString("Foo"))
   println(doubleStringList(List("Foo", "Bar", "Baz")))
}
 

As I mentioned previously, this example would be much improved by support for anonymous type functions in Scala. I am pretty sure Scala will eventually support them, as they would not require any deep changes in the implementation. They could be just implemented by desugaring to a higher-kinded type alias with a fresh name, but depending on when that desugaring is performed, it is possible that it would result in poor error messages. Supporting curried type functions is also quite desirable, but given my current knowledge of the internals, that seems like adding them will require some more elaborate changes.

I think Vincent Cremet was the first person to suggest this sort of encoding, and I vaguely recall reading about it on one of the Scala mailing lists, but I could not find the message after a little bit of time spent searching.

I also found it interesting that their dot product operator sounds more like the traditional Cartesian product type (one of each type), and their Cartesian product type sounds much more like an intersection type (the value is a structure having both types). Perhaps it is a notational holdover from previous work on species. It also seem to make for some nice concrete examples of the "locative" aspects of Girard's Ludics.

Now back to reviewing journal and ICFP papers...

So after thinking about it further, I was incorrect, and it is possible to explicitly unpack existentials in Scala. As with some other languages, it is done via pattern matching. The following example illustrates how:

val x : List[T] forSome { type T } = List(42)
val w = x match { case y : List[u] => ((z : u) => z)(y.head) }

However, in practice this functionality seems to be rather fragile. For example, the following two variations are rejected:

val x : T forSome { type T } = 42
val w = x match { case y : u => ((z : u) => z)(y) }

and

val x : List[List[T]] forSome { type T } = List(List(42))
val w = x match { case y : List[List[u]] => ((z : u) => z)(y.head.head) }

In both cases it reports that it cannot find the type u. In the second case, it could be attributed to erasure, as there is no way dynamically to guarantee that the contents of List are in turn also a List. However, the first seems reasonable, so it should probably be reported as a bug.