Apple Silicon M1: PlayFramework broken on Apple Silicon

bpossolo · February 27, 2021, 8:42pm

Has anyone gotten PlayFramework to successfully run on Apple Silicon?
I’m running the latest versions of everything and still no luck:

PlayFramework 2.8.7
scala 2.13.5
sbt 1.4.7
OpenJDK 11.0.9+11

I’m able to compile my project but run fails… even when I launch sbt with arch -x86_64 sbt

[error] java.lang.UnsatisfiedLinkError: /Users/benjamin/Library/Caches/JNA/temp/jna17546974678627290619.tmp: dlopen(/Users/benjamin/Library/Caches/JNA/temp/jna17546974678627290619.tmp, 1): no suitable image found.  Did find:
[error] 	/Users/benjamin/Library/Caches/JNA/temp/jna17546974678627290619.tmp: no matching architecture in universal wrapper
[error] 	/Users/benjamin/Library/Caches/JNA/temp/jna17546974678627290619.tmp: no matching architecture in universal wrapper
[error] 	at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
[error] 	at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442)
[error] 	at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498)
[error] 	at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694)
[error] 	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2627)
[error] 	at java.base/java.lang.Runtime.load0(Runtime.java:768)
[error] 	at java.base/java.lang.System.load(System.java:1837)
[error] 	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
[error] 	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
[error] 	at com.sun.jna.Native.<clinit>(Native.java:195)
[error] 	at io.methvin.watchservice.jna.CarbonAPI.<clinit>(CarbonAPI.java:20)
[error] 	at io.methvin.watchservice.jna.CFStringRef.toCFString(CFStringRef.java:23)
[error] 	at io.methvin.watchservice.MacOSXListeningWatchService.register(MacOSXListeningWatchService.java:127)
[error] 	at io.methvin.watchservice.WatchablePath.register(WatchablePath.java:50)
[error] 	at io.methvin.watcher.DirectoryWatcher.register(DirectoryWatcher.java:341)
[error] 	at io.methvin.watcher.DirectoryWatcher.registerAll(DirectoryWatcher.java:315)
[error] 	at io.methvin.watcher.DirectoryWatcher.<init>(DirectoryWatcher.java:176)
[error] 	at io.methvin.watcher.DirectoryWatcher$Builder.build(DirectoryWatcher.java:117)
[error] 	at play.dev.filewatch.DefaultFileWatchService.watch(DefaultFileWatchService.scala:41)
[error] 	at play.dev.filewatch.FileWatchService$$anon$1.watch(FileWatchService.scala:90)
[error] 	at play.runsupport.Reloader.<init>(Reloader.scala:443)
[error] 	at play.runsupport.Reloader$.reloader$lzycompute$1(Reloader.scala:283)
[error] 	at play.runsupport.Reloader$.play$runsupport$Reloader$$reloader$1(Reloader.scala:275)
[error] 	at play.runsupport.Reloader$.startDevMode(Reloader.scala:306)
[error] 	at play.sbt.run.PlayRun$.devModeServer$lzycompute$1(PlayRun.scala:98)
[error] 	at play.sbt.run.PlayRun$.devModeServer$1(PlayRun.scala:81)
[error] 	at play.sbt.run.PlayRun$.$anonfun$playRunTask$3(PlayRun.scala:105)
[error] 	at play.sbt.run.PlayRun$.$anonfun$playRunTask$3$adapted(PlayRun.scala:67)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] 	at sbt.Execute.work(Execute.scala:291)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:834)

For anyone else trying to develop on Apple Silicon, you’ll need to be on at least sbt 1.4.7 (see this issue about sbt support for Apple Silicon)

Related issue affecting sbt-sassify 1.5.1

bpossolo · February 27, 2021, 8:47pm

I found a partial solution that requires disabling the Play file-watcher service. It at least lets my PlayFramework server start up… but not really conducive to development without the file-watcher.

mkurz · February 27, 2021, 11:57pm

Can you try sbt 1.5.0-M2 ?
If that does not work I think you should open an issue at GitHub - gmethvin/directory-watcher: A cross-platform Java recursive directory watcher, with a JNA macOS watcher and Scala better-files integration

bpossolo · March 1, 2021, 5:06am

@mkurz
just tested upgrading to 1.5.0-M2
now compile fails…looks like the Twirl template compiler doesn’t work with sbt 1.5.0-M2

[error] java.lang.NoSuchMethodError: 'scala.tools.nsc.settings.AbsSettings$AbsSetting scala.tools.nsc.Settings.bootclasspath()'
[error] 	at play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$CompilerInstance.compiler$lzycompute(TwirlCompiler.scala:653)
[error] 	at play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$CompilerInstance.compiler(TwirlCompiler.scala:628)
[error] 	at play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$PresentationCompiler$.<init>(TwirlCompiler.scala:700)
[error] 	at play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$PresentationCompiler$.<clinit>(TwirlCompiler.scala)
[error] 	at play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$.getFunctionMapping(TwirlCompiler.scala:537)
[error] 	at play.twirl.compiler.TwirlCompiler$.generateCode(TwirlCompiler.scala:429)
[error] 	at play.twirl.compiler.TwirlCompiler$.generateFinalTemplate(TwirlCompiler.scala:487)
[error] 	at play.twirl.compiler.TwirlCompiler$.parseAndGenerateCode(TwirlCompiler.scala:259)
[error] 	at play.twirl.compiler.TwirlCompiler$.compile(TwirlCompiler.scala:200)
[error] 	at play.twirl.sbt.TemplateCompiler$.$anonfun$compile$2(TemplateCompiler.scala:60)
[error] 	at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:985)
[error] 	at scala.collection.immutable.List.foreach(List.scala:431)
[error] 	at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:984)
[error] 	at play.twirl.sbt.TemplateCompiler$.compile(TemplateCompiler.scala:50)
[error] 	at play.twirl.sbt.SbtTwirl$.$anonfun$compileTemplatesTask$1(SbtTwirl.scala:111)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] 	at sbt.Execute.work(Execute.scala:291)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:834)

mkurz · March 2, 2021, 8:55am

@bpossolo Did you run sbt clean before? Or even better, just remove the target folder in your project and try again. Let me know if the same error still occurs with sbt 1.5.0-M2

bpossolo · March 2, 2021, 5:09pm

@mkurz I tried again… deleted target and same issue

neworder:~/projects/OBFUSCATED/ui$ rm -rf target/
neworder:~/projects/OBFUSCATED/ui$ sbt
[info] welcome to sbt 1.5.0-M2 (Oracle Corporation Java 11.0.9)
[info] loading settings for project ui-build from plugins.sbt ...
[info] loading project definition from /Users/benjamin/projects/OBFUSCATED/ui/project
[info] compiling 2 Scala sources to /Users/benjamin/projects/OBFUSCATED/ui/project/target/scala-2.12/sbt-1.0/classes ...
[info] loading settings for project root from build.sbt ...
[info] set current project to OBFUSCATED (in build file:/Users/benjamin/projects/OBFUSCATED/ui/)
[info] sbt server started at local:///Users/benjamin/.sbt/1.0/server/a19349a23a1d110ba65f/sock
[info] started sbt server
[OBFUSCATED] $ compile
[error] stack trace is suppressed; run last Compile / twirlCompileTemplates for the full output
[error] (Compile / twirlCompileTemplates) java.lang.NoSuchMethodError: 'scala.tools.nsc.settings.AbsSettings$AbsSetting scala.tools.nsc.Settings.bootclasspath()'
[error] Total time: 2 s, completed Mar 2, 2021, 9:04:57 AM

francisdb · March 2, 2021, 7:01pm

Would it not be better to follow up in a github issue?

bpossolo · March 2, 2021, 11:15pm

@francisdb im not sure what the policy is for creating forum issues vs GitHub issues. the official play framework documentation says to come to the forums for support

mkurz · March 3, 2021, 1:15pm

Let’s ignore the error when using sbt 1.5.0-M2, that should not be related to the original error.
@bpossolo I think the best is to open an issue at GitHub - gmethvin/directory-watcher: A cross-platform Java recursive directory watcher, with a JNA macOS watcher and Scala better-files integration

greg · March 3, 2021, 11:20pm

According to jna/CHANGES.md at master · java-native-access/jna · GitHub, Apple Silicon support was added in JNA 5.7.0. I upgraded directory-watcher to use that version in 0.15.0. So you can try updating either JNA or directory-watcher in your plugins.sbt to see if that fixes the issue, e.g.

libraryDependencies += "io.methvin" % "directory-watcher" % "0.15.0"

bpossolo · March 4, 2021, 5:11am

@greg I added the dependency to my build.sbt
it successfully pulled in the latest version of jna and the directory watcher

[obfuscated] $ compile
[info] downloading https://repo1.maven.org/maven2/io/methvin/directory-watcher/0.15.0/directory-watcher-0.15.0.jar ...
[info] downloading https://repo1.maven.org/maven2/net/java/dev/jna/jna/5.7.0/jna-5.7.0.jar ...
[info] 	[SUCCESSFUL ] io.methvin#directory-watcher;0.15.0!directory-watcher.jar (127ms)
[info] 	[SUCCESSFUL ] net.java.dev.jna#jna;5.7.0!jna.jar (241ms)
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] compiling 82 Scala sources and 147 Java sources to /Users/benjamin/projects/obfuscated/ui/target/scala-2.13/classes ...
[success] Total time: 9 s, completed Mar 3, 2021, 9:05:22 PM

but then the same error on run

[error] java.lang.UnsatisfiedLinkError:
/Users/benjamin/Library/Caches/JNA/temp/jna3122504493045964926.tmp: dlopen(/Users/benjamin/Library/Caches/JNA/temp/jna3122504493045964926.tmp, 1): no suitable image found.  Did find:
[error]
/Users/benjamin/Library/Caches/JNA/temp/jna3122504493045964926.tmp: no matching architecture in universal wrapper
[error]
/Users/benjamin/Library/Caches/JNA/temp/jna3122504493045964926.tmp: no matching architecture in universal wrapper

bpossolo · March 4, 2021, 5:42am

Found another person experiencing the same thing (filed feb 22) so good to know its not just a me issue.

greg · March 4, 2021, 6:17am

@bpossolo you need to add it to project/plugins.sbt, because directory-watcher is used by the Play sbt plugin, not by your project code itself.

bpossolo · March 4, 2021, 8:46am

I see. I moved it into project/plugins.sbt then deleted target, ~/Library/Caches/Coursier and ~/.ivy2… and still no go

here’s my plugins.sbt file

// the play plugin
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.8.7")

// sbt eclipse plugin
// addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "5.2.4")

// plugin to assemble app into runnable fat jar
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.10")

libraryDependencies += "io.methvin" % "directory-watcher" % "0.15.0"

greg · March 4, 2021, 9:43am

@bpossolo I’m sorry I can’t be more helpful. I don’t have access to an Apple-Silicon-based machine at the moment, so it’s a little difficult for me to test.

You might also want to try using a native arm64 build of the JDK. I think this is the one: Java Download | Java 8, Java 11, Java 13 - Linux, Windows & macOS

bpossolo · March 4, 2021, 10:03am

@greg no problem. I appreciate the help anyways.

The sbt-sassify plugin is also broken because it loads a native library using JNA.
I tried several different Java versions all of which are native arm builds.
here are the virtual machines I’ve tried

16 (arm64) "UNDEFINED" - "OpenJDK 16" /opt/homebrew/Cellar/openjdk/15.0.1/libexec/openjdk.jdk/Contents/Home
15.0.2 (arm64) "Azul Systems, Inc." - "Zulu 15.29.15" /Library/Java/JavaVirtualMachines/zulu-15.jdk/Contents/Home
11.0.9 (arm64) "UNDEFINED" - "OpenJDK 11.0.9" /opt/homebrew/Cellar/openjdk@11/11.0.9/libexec/openjdk.jdk/Contents/Home

See here for my comments regarding sbt-sassify:

bpossolo · March 18, 2021, 4:56am

@mkurz provided a solution in a GitHub issue

add this to build.sbt to force a different implementation of the file watcher service

PlayKeys.fileWatchService := play.dev.filewatch.FileWatchService.jdk7(play.sbt.run.toLoggerProxy(sLog.value))

from my understanding, this uses a pure java api for file watching (on Mac OS X, the jdk implementation uses polling)… not the most optimal but better for portability which means it works on Apple Silicon!

for anyone interested in the details, the FileWatchService.jdk7() method supplies an instance of DefaultFileWatchService with isMac = false:

github.com

playframework/play-file-watch/blob/master/src/main/scala/play/dev/filewatch/DefaultFileWatchService.scala

package play.dev.filewatch

import java.io.File
import java.nio.file.FileSystems

import io.methvin.watcher.DirectoryChangeEvent
import io.methvin.watcher.DirectoryChangeListener
import io.methvin.watcher.DirectoryWatcher
import io.methvin.watchservice.MacOSXListeningWatchService

import scala.collection.JavaConverters._
import scala.util.control.NonFatal

/**
 * Implementation of the file watch service that uses a native implementation for Mac and otherwise uses the JDK's
 * WatchService implementation.
 */
class DefaultFileWatchService(logger: LoggerProxy, isMac: Boolean, disableFileHashCheck: Boolean)
    extends FileWatchService {

This file has been truncated. show original

odwrotnie · May 2, 2021, 11:33am

The similar exception occurs in Lagom applications: UnsatisfiedLinkError on Apple M1. Altering fileWatchService does not help.

greg · July 17, 2021, 4:50am

I think we identified the issue. The problem is with the fact that the sbt launcher has its own jna version that takes precedence over whatever jna version the build uses. So this needs to be fixed in sbt.io: Update JNA library for compatibility with Apple M1 · Issue #320 · sbt/io · GitHub