[Draft] A New UnsafeNulls Language Feature for Explicit Nulls

[noti0na1]

This PR removes the UncheckedNull type, and adds a new language feature unsafeNulls to explicit nulls, which allows unsafe null operations in a certain scope. In addition, a relaxed overriding rule is implemented for overriding Java classes.

Description

In the original explicit nulls design, UncheckedNull was introduced to allow member selections on nullable objects from Java code (with type T | UncheckedNull). We could define UncheckedNull as a type alias of Null or a true subtype of Null; however, we find it is difficult to implement in both ways. See #7828 for more discussion.

We decided to create a new language feature, called unsafeNulls. Inside this "unsafe" scope, all T | Null objects can be used as T, and Null keeps being a subtype of Any.

User can import scala.language.unsafeNulls to create such scope, or use -language:unsafeNulls to enable this feature globally. The following unsafe null operations will apply to all nullable types:

1. select member of T on T | Null object
2. call extension methods of T on T | Null
3. convert T1 to T2 if T1.stripAllNulls <:< T2.stripAllNulls or T1 is Null and T2 has null value after erasure

The intention of this unsafeNulls is to give users a better migration path for explicit nulls. The UncheckedNull is only limited to unsafe member selections. We think this approach can do more than UncheckedNull and in a better way.

Projects for Scala2 or regular dotty can try this by adding -Yexplicit-nulls -language:unsafeNulls to the compile options. A small number of manual modifications are expected (for example, some code relies on the facts of Null <:< AnyRef). To migrate to full explicit nulls in the future, -language:unsafeNulls can be dropped and add import scala.language.unsafeNulls only when needed.

Examples

All the examples are under -Yexplicit-nulls flag.

def f(x: String): String = ???

import scala.language.unsafeNulls

val s: String | Null = ???
val a: String = s // unsafely convert String | Null to String

val b1 = s.trim() // call .trim() on String | Null unsafely
val b2 = b1.length()

f(s).trim() // pass String | Null as an argument of type String unsafely

val c: String = null // Null to String

val d1: Array[String] = ???
val d2: Array[String | Null] = d1 // unsafely convert Array[String] to Array[String | Null]
val d3: Array[String] = Array(null) // unsafe

Without the unsafeNulls, all these unsafe operations will not be compiled.

unsafeNulls also works for extension methods and implicit search.

import scala.language.unsafeNulls

val x = "hello, world!".split(" ").map(_.length)

given Conversion[String, Array[String]] = _ => ???

val y: String | Null = ???
val z: Array[String | Null] = y 

See more examples about unsafeNulls in tests/explicit-nulls/pos/unsafe-*.scala.

An example for relaxed overriding:

abstract class J {
  abstract void f(String x);
}

// both S1 and S2 are acceptable

class S1 extends J {
  override def f(x: String) = ???
}

class S2 extends J {
  override def f(x: String | Null) = ???
}

See more examples about overriding in tests/explicit-nulls/pos/override-*.

/cc @odersky @olhotak

[olhotak]

Can you confirm that the changes to overriding of Java methods apply even without the language import? (From the code, it appears they do, and I think they should, because whether an overriding is legal is a non-local property.) If so, then perhaps the PR could be split to separate the overriding changes from the unsafe operations changes, but I'll let @odersky comment on that.

We talked earlier of having, in addition to the language import, a second language import that applies the 3 unsafe operations only when the term with the T|Null type is a call to a Java-defined method. Is that still planned for later, or did you decide against it?

I see that some of the unsafe-* tests are duplicated in pos/ and neg/, with the only difference that the pos/ version of the test has a language import. Would it make sense to avoid duplication by having an unsafe/ directory that is run twice, once with the language flag and once without?

Some of the changes (especially in the tests) appear to be whitespace-only changes. These should be reverted.

[noti0na1]

Can you confirm that the changes to overriding of Java methods apply even without the language import?

Yes, the overriding rules apply even without the unsafeNulls feature. The changes for overriding don't depend on adding unsafeNulls, but I'm not sure whether this need a separate PR.

We talked earlier of having, in addition to the language import, a second language import that applies the 3 unsafe operations only when the term with the T|Null type is a call to a Java-defined method.

Is it enough to just check the method sym is JavaDefined and apply the unsafe conversion? I can do some experiment on this after I fix some other bugs.

I see that some of the unsafe-* tests are duplicated in pos/ and neg/, with the only difference that the pos/ version of the test has a language import.

Yes, I think these tests can be combined.

We are also thinking about making Null a subtype of AnyVal. Some old discussion can be found here: #6344 (comment)

I am also doing same experiment on this. It seems I need to modify the codegen part.

@olhotak

[olhotak]

We talked earlier of having, in addition to the language import, a second language import that applies the 3 unsafe operations only when the term with the T|Null type is a call to a Java-defined method.

Is it enough to just check the method sym is JavaDefined and apply the unsafe conversion? I can do some experiment on this after I fix some other bugs.

Yes, I think so.

You apply rules 1, 2, and 3 (from your description of this PR) only when the tree whose type is nullable is a call to a Java method, so presumably an Apply whose fun has a symbol that is JavaDefined.

We are also thinking about making Null a subtype of AnyVal. Some old discussion can be found here: #6344 (comment)

I am also doing same experiment on this. It seems I need to modify the codegen part.

I think it's worth experimenting with, though in a separate PR.

[noti0na1]

I will close this PR, because I moved the source branch to dotty-staging.