Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start benchmarking #29

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,5 @@ metals.sbt
async-profiler/
diesel-samples/jvm/profile*

*gpcreds.json
*gpcreds.json
/benchmark/jvm/*/profile.jfr
47 changes: 47 additions & 0 deletions benchmark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Benchmarking and Profiling

## Scala JS

Build in `sbt` using
```
benchmarkJS/fastOptJS
```
And open in [index.html](./index.html) a browser.

Use devtool profiling ...

## Scala JVM

Run from `sbt` using
```
benchmark/jmh:run -i 3 -wi 3 -f1 -t1
benchmark/jmh:run
benchmark/jmh:run -h
```

### Profiling hints:
#### Java Flight Recorder
```
benchmark/jmh:run -prof jfr:help
benchmark/jmh:run -i 2 -wi 2 -f1 -t1 -prof jfr
```

This will write `profile.jfr` files under _jvm/_,
which can be analyzed using IntelliJ.


#### async-profiler (needs to be installed separately)
```
benchmark/jmh:run -prof async:help
benchmark/jmh:run -i 2 -wi 2 -f1 -t1 -prof async
```

## Adding a benchmark case

- add a function to `diesel.benchmark.BenchmarkCases` in _shared_ sources.
- integrate with JS in _js_ sources: `diesel.benchmark.DieselBenchmark`
- integrate with JVM in _jvm_ sources: `diesel.benchmark.DieselBenchmark`

## Links
- https://github.com/japgolly/scalajs-benchmark
- https://github.com/sbt/sbt-jmh
12 changes: 12 additions & 0 deletions benchmark/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<!DOCTYPE html>
<html>
<head lang="en">
<title>scalajs-benchmarks</title>
</head>
<body>
<div id="body">Loading...</div>
<script type="text/javascript" src="js/target/scala-2.13/diesel-core-benchmark-jsdeps.js"></script>
<script type="text/javascript" src="js/target/scala-2.13/diesel-core-benchmark-fastopt.js"></script>
<script type="text/javascript">main();</script>
</body>
</html>
35 changes: 35 additions & 0 deletions benchmark/js/src/main/scala/diesel/benchmark/DieselBenchmark.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright 2018 The Diesel Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package diesel.benchmark

import japgolly.scalajs.benchmark._
import japgolly.scalajs.benchmark.gui._

import BenchmarkCases._

object DieselBenchmark {
val suite = GuiSuite(
Suite("Diesel Benchmarks")(
Benchmark("Simple BMD") {
parseSimpleBmd
},
Benchmark("Some Glsl") {
parseSomeGlsl
}
)
)
}
28 changes: 28 additions & 0 deletions benchmark/js/src/main/scala/diesel/benchmark/Main.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
* Copyright 2018 The Diesel Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package diesel.benchmark

import org.scalajs.dom.document
import japgolly.scalajs.benchmark.gui.BenchmarkGUI

object Main {

def main(): Unit = {
val body = document getElementById "body"
BenchmarkGUI.renderSuite(body)(DieselBenchmark.suite)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/*
* Copyright 2018 The Diesel Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package diesel.benchmark

import diesel.benchmark.BenchmarkCases._
import org.openjdk.jmh.annotations.Benchmark

class DieselBenchmark {
@Benchmark
def simpleBMD(): Unit = {
parseSimpleBmd
}

@Benchmark
def someGlsl(): Unit = {
parseSomeGlsl
}

}
149 changes: 149 additions & 0 deletions benchmark/shared/src/main/scala/diesel/benchmark/BenchmarkCases.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
/*
* Copyright 2018 The Diesel Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package diesel.benchmark

import diesel.samples.glsl.Glsl
import diesel.samples.jsmodeldsl.BmdDsl
import diesel.{AstHelpers, Dsl}

object BenchmarkCases {

def parseSimpleBmd: Unit = {
val text =
"""|start with a Foo.
|a Foo is a concept.
|a Gnu is a xx.
|""".stripMargin
parseSome(BmdDsl, Some(BmdDsl.aCompileUnit))(text)
}

def parseSomeGlsl: Unit = {
val text =
"""void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|
|float x = 12;
|
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|
|void mainImage( out vec4 fragColor, in vec2 fragCoord )
|{
| vec2 uv = fragCoord/iResolution.xy;
|
| if (true) {
| return 12;
| }
|
| vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
|
| fragColor = vec4(col,1.0);
|}
|""".stripMargin
parseSome(Glsl, Some(Glsl.a))(text)
}

private def parseSome(dsl: Dsl, axiom: Option[Dsl.Axiom[_]] = None)(text: String): Unit = {
AstHelpers.parse(dsl, text, axiom)
}

// use main with IntelliJ profiler
def main(args: Array[String]): Unit = {
parseSimpleBmd
parseSimpleBmd
}
}
37 changes: 34 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import sbt.{CrossVersion, ThisBuild}
import sbtcrossproject.CrossPlugin.autoImport.crossProject

import scala.sys.process._
import org.scalajs.linker.interface.ModuleInitializer.mainMethod

val scalaVersion2 = "2.13.10"
// val scalaVersion3 = "3.2.1"
Expand Down Expand Up @@ -44,7 +43,15 @@ addCommandAlias("testJS", "all dieselJS/test samplesJS/test")

lazy val root = project
.in(file("."))
.aggregate(diesel.jvm, diesel.js, samples.jvm, samples.js, samplesBundle)
.aggregate(
diesel.jvm,
diesel.js,
samples.jvm,
samples.js,
samplesBundle,
benchmark.jvm,
benchmark.js
)
.settings(commonSettings)
.settings(sonatypeSettings)
.settings(copyrightSettings)
Expand Down Expand Up @@ -142,3 +149,27 @@ lazy val samplesBundle = project
.settings(commonSettings)
.settings(copyrightSettings)
.dependsOn(samples.js)

lazy val benchmark = crossProject(JSPlatform, JVMPlatform)
.withoutSuffixFor(JVMPlatform)
.settings(commonSettings)
.settings(copyrightSettings)
.settings(
name := "diesel-core-benchmark",
publish / skip := true,
test := {}
)
.settings(sharedSettings_scalac)
.settings(sharedSettings_lint)
.enablePlugins(JSDependenciesPlugin, JmhPlugin)
.jsSettings(
libraryDependencies ++= Seq(
"com.github.japgolly.scalajs-benchmark" %%% "benchmark" % "0.10.0",
"org.scala-js" %%% "scalajs-dom" % "2.5.0"
),
scalaJSUseMainModuleInitializer := true,
Compile / scalaJSMainModuleInitializer := Some(mainMethod("diesel.benchmark.Main", "main")),
scalaJSLinkerConfig ~= { _.withSourceMap(true) },
packageJSDependencies / skip := false
)
.dependsOn(samples)
2 changes: 2 additions & 0 deletions project/plugins.sbt
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
addSbtPlugin("org.portable-scala" % "sbt-scalajs-crossproject" % "1.3.1")
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "1.13.1")
addSbtPlugin("org.scala-js" % "sbt-jsdependencies" % "1.0.2")
addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.5.0")
addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.10.4")
addSbtPlugin("de.heikoseeberger" % "sbt-header" % "5.9.0")
addSbtPlugin("com.github.sbt" % "sbt-ci-release" % "1.5.12")
addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.4.4")

addSbtPlugin(
"com.ibm.cloud.diesel" % "diesel-i18n-plugin" % "0.6.0"
Expand Down