That was a longer hiatus than I had intended. Partly because not everything went according to plan.

My original plan was that, upon returning from my vacation, I would spend my remaining time at EPFL writing a technical report explaining everything I knew about the problems with Scala Classic. Instead, on my first day back in the office, Martin came by with a draft of a new formal system, DOT (for Dependent Object Types), that he came up with while he was on vacation. After about four weeks I managed to root out pretty much all of the obvious problems with DOT, but another four weeks was not enough to get the proofs and the metatheory into a state that we were happy with. I am not really sure what will happen with DOT in the long term.

There are a few clever things about DOT that avoid some of the problems I encountered with Scala Classic. For example, Scala Classic only had intersection types while DOT has both intersection and union types, which solves some problems with computing the members of types.  However, with Scala Classic I was trying to model a minimal subset Scala language as it exists, without adding any new features that many never be supported by the full language. I have no idea if we will see union types in the actual Scala implementation anytime soon.

The other thing DOT does is give up the goal of trying to be a minimal subset of Scala and throws out quite a few important things from a user perspective. For example, there are no methods (only λ-abstractions), no existentials, no built-in inheritance or mix-in composition (though you can encode it by hand), and object fields must be syntactically values.

This last restriction is particularly important because it solves by fiat the problems that arose in Scala Classic from using paths that have an empty type in dependent type projections. However, it also means that you may not be able to directly encode some Scala programs into DOT without an effect or termination analysis. Therefore, while DOT has the potential to be a useful intermediary step, there will still be more work to be done to provide a semantics for a more realistic subset of the Scala language.

I have been in Copenhagen for a little over a month now, but I am not really ready to comment on the research I have been doing here yet.  So in the meantime I'll probably focus more on fonts and typography.

Given that the formalization of Scala Classic has ground to a halt, for reasons I may go into later, I spent part of today hacking on the Scala compiler itself to add support for singleton literals. Currently, Scala allows singleton types for stable identifiers. My modification allows literals to be used in singleton types. I can't promise that it will be in the forthcoming Scala 2.7.2 release, but I would like it to be.

Overall it was far less work than I was expecting.  Scala already internally supports what it calls "constant types", there is just no way to write them in Scala source code presently.  Consequently, most of the effort was in extending the parser.

Given my modifications, it is now possible to write code like the following:

 
scala> val x : "foo".type = "foo"
x: "foo".type = foo
 

What I was not expecting was that out-of-the-box things like the following would work:

 
scala> val x : "foobar".type = "foo" + "bar"
x: "foobar".type = foobar
scala> val y : 10.type = 2 * 5
y: 10.type = 10
scala> def frob(arg : 10.type) : 6.type = arg - 4
frob: (10.type)6.type
 

Unfortunately the excitement soon passes when you realize all the things you can't do with singleton literals (yet). Even if we turn on the experimental dependent method support, you can't write things like

 
def add(arg : Int) : (arg + 5).type = arg + 5
 

because these are exactly what they are called, singleton literals, not full-blown dependent types.

One cute example, based on a use suggested by Sean McDirmid, would be that some people might do something like the following with implicits:

 
implicit stringToColor(arg : String) : java.awt.Color = java.awt.Color.getColor(arg)
 

However, with singleton literals you can tighten it up by ensuring that for some strings it will never fail:

 
implicit redToColor(arg : "red".type) : java.awt.Color = java.awt.Color.RED
implicit blueToColor(arg : "blue".type) : java.awt.Color = java.awt.Color.BLUE
 

Happily, the type inferencer already chooses the most specific implicit conversions.

In any event, they will hopefully become more useful as the type system continues to grow. I am also sure someone will probably come up with a clever use for them that hasn't occurred to me yet. If so, let me know.

Several weeks ago I decided that instead of calling the core calculus that I have been working on Featherweight Scala, which could be confusing given that there is already a different calculus with that name, that I would call it Scala Classic.

I had hoped to submit a paper on Scala Classic to POPL 2009, but it has been taking too long to work out the metatheory. Partly because I am developing a mechanized proof of type soundness in Twelf, and on my first attempt at representing heaps I think tried to be too clever.  However, a more traditional treatment of heaps leads to the need to explicitly represent typing contexts, rather than the implicit treatment of contexts more commonly used in Twelf.

This afternoon I finally finished the proof of the theorem that is traditionally called "substitution".  However, the proof of type preservation will also require another theorem formalizing a substitution-like property for singleton types.  I am hoping that I can prove that theorem more quickly now that I've built up all the explicit context theory.  I have not thought much about whether there are any unusual theorems needed to prove progress.

In any event, I would ideally have the proofs finished by the beginning of August.  My current plan is to try submitting the paper to PLDI (which is even in Europe this year), unless some obviously better venue comes to my attention.

One of the reasons I would really like to have the proofs done by August is that I will be teaching a PhD level course here at EPFL this fall on mechanized sepcifications and proofs, with a strong emphasis on deductive systems. It also looks like I may fill in some for a masters level course based on TAPL.  So between those two things, I am not going to have nearly as much time for research as would be ideal.

Following on my earlier entry on modules in Scala, I'll give an encoding of Standard ML style functors here. You can get a pretty close approximation by using class constructor arguments. However, I am going to cheat a little to get the closest encoding I think is possible by using the experimental support for dependent method types. You can get this by running scala or scalac with the option -Xexperimental. It works okay at least some of the time, but no one has the time at the moment to commit to getting it in shape for general consumption.

So here is my example of how the encoding works. First, the SML version:

 
signature Eq = sig
  type t
  val eq: t -> t -> bool
end
 
signature RicherEq = sig
  include Eq
  val neq: t -> t -> bool
end
 
functor mkRicherEq(arg : Eq) :> RicherEq where type t = arg.t = struct
  type t = arg.t
  val eq = arg.eq
  fun neq x y = not (eq x y)
end
 

We can transliterate this example into Scala as:

 
type Eq = {
  type T
  def eq(x: T, y: T): Boolean
}
 
type RicherEq = {
  type T
  def eq(x: T, y: T): Boolean
  def neq(x: T, y: T): Boolean
}
 
def mkRicherEq(arg: Eq) : RicherEq { type T = arg.T } = new {
  type T = arg.T
  def eq(x: T, y: T) = arg.eq(x, y)
  def neq(x: T, y:T) = !eq(x, y)
}
 

The only problem I discovered is that it is not possible to define RicherEq in terms of Eq as we could in SML:

 
scala> type RicherEq = Eq { def neq(x: T, y: T): Boolean }
<console>:5: error: Parameter type in structural refinement may
not refer to abstract type defined outside that same refinement
       type RicherEq = Eq { def neq(x: T, y: T): Boolean }
                                       ^
<console>:5: error: Parameter type in structural refinement may
not refer to abstract type defined outside that same refinement
       type RicherEq = Eq { def neq(x: T, y: T): Boolean }
                                             ^
 

Why this restriction exists I don't know. In fact, this sort of refinement should work in the current version of Featherweight Scala, so perhaps it can be lifted eventually.

I still need to think about higher-order functors, and probably spend a few minutes researching existing proposals. I think this is probably something that cannot be easily supported in Scala if it will require allowing method invocations to appear in paths. However, off hand that only seems like it should be necessary for applicative higher-order functors, but again I definitely need to think it through.

I just saw a thread on Lambda the Ultimate where I think the expressive power of Scala in comparison to Standard ML's module system was misrepresented. I don't want to go into all of the issues at the moment, but I figured out would point out that you can get the same structural typing, opaque sealing, and even the equivalent of SML's where type clause.

For example, consider the following SML signature:

 
signature Nat = sig
  type t
  val z: t
  val s: t -> t
end
 

This signature can be translated in to Scala as:

 
type Nat = {
  type T
  val z: T
  def s(arg: T): T
}
 

It is then possible to create an implementation of this type, and opaquely seal it (hiding the definition of T). In SML:

 
structure nat :> Nat = struct
  type t = int
  val z = 0
  fun s n = n + 1
end
 

In Scala:

 
val nat : Nat = new {
  type T = Int
  val z = 0
  def s(arg: Int) = arg + 1
}
 

In many cases when programming with SML modules it is necessary or convenient to give a module that reveals the definition of an abstract type. In the above example, this can be done by adding a where type clause to the first line:

 
structure nat :> Nat where type t = int = struct
...
 

We can do the same thing in Scala using refinements:

 
val nat : Nat { type T = Int } = new {
...
 

Great, right? Well, almost. The problem is that structural types are still a bit buggy in Scala compiler at present. So, while the above typechecks, you can't quite use it yet:

 
scala> nat.s(nat.z)
java.lang.NoSuchMethodException: $anon$1.s(java.lang.Object)
        at java.lang.Class.getMethod(Class.java:1581)
        at .reflMethod$Method1(<console>:7)
        at .<init>(<console>:7)
        at .<clinit>(<console>)
        at RequestResult$.<init>(<console>:3)
        at RequestResult$.<clinit>(<console>)
        at RequestResult$result(<console>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflec...
 

There were some issues raised about how faithful an encoding of SML functors, and well-known extensions for higher-order functors, one can get in Scala. Indeed, off the top of my head it is not entirely clear. So I need to think more about that before I write some examples.

With everything else I've been having to deal with the past couple months, I have not made nearly as much progress on Featherweight Scala as I would have liked. Fortunately, in the past two weeks I have finally started to get back on track.

Probably what makes working out the design of Featherweight Scala so difficult is that I am trying my best to commit to it being a valid subset of Scala. It seems like whenever I try to simplify one part of the language, it just introduces a complication elsewhere. Still, I think so far that I have been able to ensure that it is a subset with two exceptions.

The first exception is that some programs in Featherweight Scala will diverge in cases where the Scala compiler will generate a terminating program. This is because in programs that attempt to access an uninitialized field, Scala will return null, while Featherweight Scala diverges. Since the program many never attempt to use the null value, it may simple terminate without raising a NullPointerException.

The second exception arises because type checking in Featherweight Scala's type system is most likely undecidable. Given that the design is in flux, I haven't precisely verified this, but it seems quite plausible given its expressive power is similar to languages where type checking is known to be undecidable. Because type checking is undecidable, and the Scala language implementation attempts to have its type checking phase be terminating, there should be some programs that can be shown to be well-typed in Featherweight Scala that the Scala compiler will reject.

I think the other thing I've determined is that I do not understand F-bounded quantification as well as I thought I did. At least, I find myself worried about the cases where the typing rules presently require  templates, that have not yet been verified to be well-formed, to be introduced as hypotheses while verifying their own correctness. So I need to do some additional reading to see how some other languages deal with this issue. Certainly I can put my mind at ease by making the type system more restrictive, but I do not want to be so restrictive that it is no longer reasonable to call the language Featherweight Scala.

Update: Okay, so after looking at Featherweight Generic Java and νObj again more closely, they both do essentially the same kind of thing. In FGJ, the class table is well-formed if all its classes are well-formed, but checking a class's well-formedness may require presupposing that that class tale is well-formed. In νObj, the well-formedness of recursive record types is checked under the assumption that the record's self-name has the type of the record. Certainly the same sort of things come up when verifying the well-formedness of recursive functions and recursive types, but at least there what is being assumed is at a higher level of abstraction: when checking a recursive function you assume that the function has some well-formed type, or when checking a recursive type you assume that the type has some well-formed kind.

I've been meaning to write this up for a while, but it seems like there has always been something else I really ought to be doing. So I expect this will be a bit more terse than I might like. Anyway, when I wrote about encoding higher-rank universal quantification in Scala back in March, I used a rather elaborate scheme involving the use of Scala's first-class existential quantifiers. While this was the point of the exercise, surprisingly, no one called me on the fact that if you just want higher-rank impredicative polymorphism in Scala, there is a much simpler encoding. Maybe it was obvious to everyone or no one read closely enough to think of raising the issue. So today, I'll explain the better way to encode them.

First we can define an infinite family of traits to represent n-ary universal quantification, much like Scala represents n-ary functions types:

 
trait Univ1[Bound1,Body[_]] {
   def Apply[T1<:Bound1] : Body[T1]
}
 
trait Univ2[Bound1,Bound2,Body[_,_]] {
   def Apply[T1<:Bound1,T2<:Bound2] : Body[T1,T2]
}
 
// ... so on for N > 2
 

Really, the key to this encoding is the use of higher-kinded quantification to encode the binding structure of the quantifier.

Now it is possible to write some examples similar to what I gave previously, but more concisely:

 
object Test extends Application {
   def id[T](x : T) = x
 
   type Id[T] = T => T
   val id = new Univ1[Any,Id]
     { def Apply[T <: Any] : Id[T] = id[T] _ }
 
   val idString = id.Apply[String]
   val idStringList = id.Apply[List[String]]
 
   println(idString("Foo"))
   println(idStringList(List("Foo", "Bar", "Baz")))
 
   type Double[T] = T => (T, T)
   val double = new Univ1[Any,Double]
     { def Apply[T <: Any] : Double[T] = (x : T) => (x, x) }
 
   val doubleString = double.Apply[String]
   val doubleStringList = double.Apply[List[String]]
 
   println(doubleString("Foo"))
   println(doubleStringList(List("Foo", "Bar", "Baz")))
}
 

As I mentioned previously, this example would be much improved by support for anonymous type functions in Scala. I am pretty sure Scala will eventually support them, as they would not require any deep changes in the implementation. They could be just implemented by desugaring to a higher-kinded type alias with a fresh name, but depending on when that desugaring is performed, it is possible that it would result in poor error messages. Supporting curried type functions is also quite desirable, but given my current knowledge of the internals, that seems like adding them will require some more elaborate changes.

I think Vincent Cremet was the first person to suggest this sort of encoding, and I vaguely recall reading about it on one of the Scala mailing lists, but I could not find the message after a little bit of time spent searching.

I spent some time last night looking more closely at Carette and Uszkey's new paper on species, as mentioned recently on Lambda the Ultimate. I mostly concentrated on the first few sections and the Haskell bit at the end. They say many tantalizing things, and I had kind of hoped the punchline would be something like "here is how you can write code in a beautifully functional style over graphs and cyclic structures" (assuming I understood the potential of species correctly).  Perhaps in a follow up paper.

I also found it interesting that their dot product operator sounds more like the traditional Cartesian product type (one of each type), and their Cartesian product type sounds much more like an intersection type (the value is a structure having both types). Perhaps it is a notational holdover from previous work on species. It also seem to make for some nice concrete examples of the "locative" aspects of Girard's Ludics.

Now back to reviewing journal and ICFP papers...

I finally got the journal version »Generalizing Parametricity Using Information Flow« out for consideration the week before last. I really should have a copy up on my website, but I figured I would wait until I finish and push out the revised version of my website that I've been working on. So, in the meantime, you can get it from Stephanie's web site.

However, I've mostly put work on my revised website on hold now that I am ramping up production of my ICFP submission based on InforML. Last Sunday I was feeling like I just wasn't going to be able to fit everything I needed to say in twelve pages of SIGPLAN style. However, think I've come up with a new take on presenting the material that should get me close enough to the page limit, that I can probably successively optimize the prose and formatting to get it all in. I'll see.  Regardless I will be glad to get all of this old material out of the way, so I can continue focusing on new and exiciting (at least to a programming languages researcher) ideas.

From the fact that the ICFP website still does not list a contest coordinator, I am tending to believe that there will not be a programming contest this year, which is kind of disappointing.