Skip to content

Commit 47b91b1

Browse files
committed
Merge branch 'scala3doc/original-code' into scala3doc/migration
2 parents 4ad3ffb + c883fe3 commit 47b91b1

File tree

380 files changed

+36904
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

380 files changed

+36904
-0
lines changed

.github/workflows/scala3doc.yaml

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
name: CI for dotty-dokka
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
pull_request:
8+
jobs:
9+
build:
10+
runs-on: ubuntu-latest
11+
timeout-minutes: 20
12+
steps:
13+
- name: set right cwd
14+
run: cd scala3doc
15+
16+
- uses: actions/checkout@v2
17+
- run: git fetch --prune --unshallow --tags
18+
19+
- name: Cache Coursier
20+
uses: actions/cache@v1
21+
with:
22+
path: ~/.cache/coursier
23+
key: sbt-coursier-cache
24+
- name: Cache SBT
25+
uses: actions/cache@v1
26+
with:
27+
path: ~/.sbt
28+
key: sbt-${{ hashFiles('**/build.sbt') }}
29+
30+
- name: Set up JDK 11
31+
uses: actions/setup-java@v1
32+
with:
33+
java-version: 11
34+
35+
- name: Compile and test
36+
run: sbt compile test
37+
38+
- name: Locally publish self
39+
run: sbt publishLocal
40+
41+
- name: Generate test documentation
42+
run: sbt generateSelfDocumentation
43+
44+
- name: Generate documentation for dotty library
45+
run: sbt generateDottyLibDocumentation
46+
47+
- name: Generate documentation with SBT plugin
48+
run: sbt example-project/doc
49+
50+
- name: Configure AWS Credentials
51+
uses: aws-actions/configure-aws-credentials@v1
52+
with:
53+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
54+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
55+
aws-region: eu-central-1
56+
57+
- name: Publish all
58+
run: |
59+
dest=s3://scala3doc/$([ $GITHUB_REF = "master" ] && echo master || echo pr-$(echo $GITHUB_REF | cut '-d/' -f3))
60+
aws s3 rm $dest
61+
aws s3 sync output $dest
62+
63+
- name: Update gh-pages
64+
run: |
65+
if [ $GITHUB_REF = "refs/heads/master" ]; then
66+
git config --global user.email "$(git log -1 --format=%ae)"
67+
git config --global user.name "$(git log -1 --format=%an)"
68+
git subtree add --prefix=our-site origin gh-pages
69+
rm -r our-site/*
70+
cp -r output/self/* our-site
71+
git add our-site
72+
git commit -m "$(git log -1 --pretty=%B)"
73+
git subtree push --prefix=our-site origin gh-pages
74+
fi
75+

.vscode/settings.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"files.watcherExclude": {
3+
"**/target": true
4+
}
5+
}

scala3doc/.gitignore

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
*.class
2+
*.log
3+
libs/
4+
5+
# sbt specific
6+
.cache
7+
.history
8+
.lib/
9+
dist/*
10+
target/
11+
lib_managed/
12+
src_managed/
13+
project/boot/
14+
project/plugins/project/
15+
16+
# Scala-IDE specific
17+
.scala_dependencies
18+
.worksheet
19+
.bsp
20+
21+
# Intellij
22+
.idea
23+
24+
#integration tests dir
25+
/test-ws
26+
27+
# to test deploy
28+
private-*
29+
repo
30+
31+
.metals
32+
metals.sbt
33+
.bloop
34+
.dotty-ide*
35+
36+
# custom things
37+
output
38+
39+
.vscode
40+

scala3doc/.scalafmt.conf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
version = "2.6.3"

scala3doc/README.md

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Scala3doc
2+
3+
Scala3doc (name subject to change) is the documentation tool for
4+
[Dotty](https://github.com/lampepfl/dotty), which is scheduled to become
5+
Scala 3. It's based on [Dokka](https://github.com/Kotlin/dokka), the
6+
documentation tool for Kotlin. It uses Tasty-Reflect to access definitions,
7+
which is an officially supported way to access Dotty's perspective of a
8+
codebase.
9+
10+
We're aiming to support all the features Scaladoc did, plus new and exciting ones such as:
11+
12+
- Markdown syntax!
13+
- displaying project and API documentation together on one site!
14+
- and more!
15+
16+
## Running the project
17+
18+
Use the following commands to generate documentation for this project and for Dotty, respectively:
19+
20+
```
21+
sbt generateSelfDocumentation
22+
sbt generateDottyLibDocumentation
23+
```
24+
25+
To actually view the documentation, the easiest way is to run the following in project root:
26+
27+
```
28+
cd output
29+
python3 -m http.server 8080
30+
```
31+
32+
And afterwards point your browser to `http://localhost:8080/self` or
33+
`http://localhost:8080/stdLib` for this project and for Dotty documentation
34+
respectively.
35+
36+
It's not strictly necessary to go through an HTTP server, but because of CORS
37+
the documentation won't work completely if you don't.
38+
39+
## Developing
40+
41+
At least two of our contributors use [Metals](https://scalameta.org/metals/) to
42+
work on the project.
43+
44+
For every PR, we build documentation for Scala3doc and Dotty. For example, for
45+
PR 123 you can find them at:
46+
47+
+ https://scala3doc.s3.eu-central-1.amazonaws.com/pr-123/self/main/index.html
48+
+ https://scala3doc.s3.eu-central-1.amazonaws.com/pr-123/stdLib/main/index.html
49+
50+
Note that these correspond to the contents of `output` directory - that's
51+
precisely what they are.
52+
53+
You can also find the result of building the same sites for latest `master` at:
54+
55+
+ https://scala3doc.s3.eu-central-1.amazonaws.com/pr-master/self/main/index.html
56+
+ https://scala3doc.s3.eu-central-1.amazonaws.com/pr-master/stdLib/main/index.html
57+
58+
### Testing
59+
60+
Most tests rely on comparing signatures (of classes, methods, objects etc.) extracted from the generated documentation
61+
to signatures found in source files. Such tests are defined using [MultipleFileTest](src/test/scala/dotty/dokka/MultipleFileTest.scala) class
62+
and its subtypes (such as [SingleFileTest](src/test/scala/dotty/dokka/SingleFileTest.scala))
63+
64+
WARNING: As the classes mentioned above are likely to evolve, the description below might easily get out of date.
65+
In case of any discrepancies rely on the source files instead.
66+
67+
`MultipleFileTest` requires that you specify the names of the files used to extract signatures,
68+
the names of directories containing corresponding TASTY files
69+
and the kinds of signatures from source files (corresponding to keywords used to declare them like `def`, `class`, `object` etc.)
70+
whose presence in the generated documentation will be checked (other signatures, when missing, will be ignored).
71+
The mentioned source files should be located directly inside `src/main/scala/tests` directory
72+
but the file names passed as parameters should contain neither this path prefix nor `.scala` suffix.
73+
The TASTY folders are expected to be located in `target/${dottyVersion}/classes/tests` (after successful compilation of the project)
74+
and similarly only their names relative to this path should be provided as tests' parameters.
75+
For `SingleFileTest` the name of the source file and the TASTY folder are expected to be the same.
76+
77+
By default it's expected that all signatures from the source files will be present in the documentation
78+
but not vice versa (because the documentation can contain also inherited signatures).
79+
To validate that a signature present in the source does not exist in the documentation
80+
(because they should be hidden from users) add `//unexpected` comment after the signature in the same line.
81+
This will cause an error if a signature with the same name appears in the documentation
82+
(even if some elements of the signature are slightly different - to avoid accidentally passing tests).
83+
If the signature in the documentation is expected to slightly differ from how it's defined in the source code
84+
you can add a `//expected: ` comment (also in the same line and followed by a space) followed by the expected signature.
85+
Alternatively you can use `/*<-*/` and `/*->*/` as opening and closing parentheses for parts of a signature present in the source but undesired in the documentation (at least at the current stage of development), e.g.
86+
87+
```
88+
def foo/*<-*/()/*->*/: Int
89+
```
90+
91+
will make the expected signature be
92+
93+
```
94+
def foo: Int
95+
```
96+
97+
instead of
98+
99+
```
100+
def foo(): Int
101+
```
102+
103+
104+
Because of the way how signatures in source are parsed, they're expected to span until the end of a line (including comments except those special ones mentioned above, which change the behaviour of tests) so if a definition contains an implementation, it should be placed in a separate line, e.g.
105+
106+
```
107+
def foo: Int
108+
= 1
109+
110+
class Bar
111+
{
112+
//...
113+
}
114+
```
115+
116+
Otherwise the implementation would be treated as a part of the signature.
117+
118+
## Roadmap
119+
120+
1. Publish an initial version of the tool together with an SBT plugin
121+
1. Replace Dottydoc as the dedicated tool for documenting Dotty code
122+
123+
This includes:
124+
+ supporting Dotty's doc pages
125+
+ releasing together with Dotty as the dedicated documentation tool
126+
127+
1. Support all kinds of Dotty definition and generate documentation for the
128+
standard library
129+
1. Reach feature parity with Scaladoc
130+
131+
## Contributing
132+
133+
We're happy that you'd like to help us!
134+
135+
We have two issue labels you should take a look at: `good first issue` and
136+
`self-contained`. First is easy pickings: you'll be able to contribute without
137+
needing to dive too deep into the project. Second is reverse: it's an issue
138+
that's you may find interesting, complex and self-contained enough that you can
139+
continue chipping away at it without needing to worry too much about merge
140+
conflicts.
141+
142+
To contribute to the project with your code, fork this repo and create a pull request from a fresh branch from there.
143+
To keep the history of commits clean, make sure your commits are squashed into one
144+
and all your changes are applied on top of the latest master branch (if not - rebase on it instead of merging it).
145+
Make sure all the tests pass (simply run `sbt test` to verify that).
146+
147+
## FAQ
148+
149+
### Why depend on Dokka?
150+
151+
We have two primary reasons for depending on Dokka. One of them is division of
152+
labour - Dokka already has a team of maintainers, and it supports an excellent
153+
API which already allowed us to quite easily generate documentation with it. By
154+
depending on Dokka, we will be able to share a large portion of the maintenance
155+
burden. The second reason is very pragmatic - on our own, it'd be difficult for
156+
us to reach even feature parity with Scaladoc, simply because of workforce
157+
constraints. Meanwhile, Dokka maintainers from VirtusLab reached out to us with
158+
an offer of help, which we were happy to take.
159+
160+
### Why use TASTy?
161+
162+
A documentation tool needs to access compiler information about the project - it
163+
needs to list all definitions, resolve them by name, and query their members.
164+
Tasty Reflect is the dedicated way in Scala 3 of accessing this information.

scala3doc/build.sbt

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
// re-expose subproject settings
2+
val `example-project` = ExampleProject.`example-project`
3+
4+
val dottyVersion = "0.27.0-RC1"
5+
val dokkaVersion = "1.4.10.2"
6+
val flexmarkVersion = "0.42.12"
7+
val jacksonVersion = "2.9.8"
8+
val scalaTagsVersion = "0.9.1"
9+
val dokkaSiteVersion = "0.1.9"
10+
11+
libraryDependencies ++= Seq(
12+
"org.jetbrains.dokka" % "dokka-test-api" % dokkaVersion % "test", // TODO move testing utils to dokka-site
13+
"com.virtuslab.dokka" % "dokka-site" % dokkaSiteVersion,
14+
15+
"ch.epfl.lamp" %% "dotty-tasty-inspector" % dottyVersion,
16+
"ch.epfl.lamp" %% "dotty-compiler" % dottyVersion,
17+
"ch.epfl.lamp" %% "dotty-library" % dottyVersion,
18+
"org.scala-sbt" % "io_2.13" % "1.3.4",
19+
20+
"com.vladsch.flexmark" % "flexmark-all" % flexmarkVersion,
21+
"com.lihaoyi" % "scalatags_2.13" % scalaTagsVersion,
22+
"nl.big-o" % "liqp" % "0.6.7",
23+
"args4j" % "args4j" % "2.33",
24+
"com.novocode" % "junit-interface" % "0.11" % "test",
25+
)
26+
27+
resolvers += Resolver.jcenterRepo
28+
resolvers += Resolver.bintrayRepo("kotlin", "kotlin-dev")
29+
resolvers += Resolver.bintrayRepo("virtuslab", "dokka")
30+
resolvers += Resolver.mavenLocal
31+
32+
lazy val root = project
33+
.in(file("."))
34+
.settings(
35+
name := "scala3doc",
36+
version := "0.1.1-SNAPSHOT",
37+
scalaVersion := dottyVersion
38+
)
39+
40+
val generateSelfDocumentation = inputKey[Unit]("Generate example documentation")
41+
generateSelfDocumentation := {
42+
run.in(Compile).fullInput(" -o output/self -t target/scala-0.27/classes -d documentation -n scala3doc -s src/main/scala=https://github.com/lampepfl/scala3doc/tree/master/src/main/scala#L").evaluated // TODO #35 proper sbt integration
43+
}
44+
45+
// Uncomment to debug dokka processing (require to run debug in listen mode on 5005 port)
46+
// javaOptions.in(run) += "-agentlib:jdwp=transport=dt_socket,server=n,address=localhost:5005,suspend=y"
47+
48+
fork.in(run) := true
49+
// There is a bug in dokka that prevents parallel tests withing the same jvm
50+
fork.in(test) := true
51+
Test / parallelExecution := false
52+
53+
scalacOptions in Compile += "-language:implicitConversions"
54+
55+
Compile / mainClass := Some("dotty.dokka.Main")
56+
57+
// hack, we cannot build documentation so we need this to publish locally
58+
publishArtifact in (Compile, packageDoc) := false
59+
60+
// TODO #35 proper sbt integration
61+
val generateDottyLibDocumentation = taskKey[Unit]("Generate documentation for dotty lib")
62+
generateDottyLibDocumentation := Def.taskDyn {
63+
val dotttyLib = fullClasspath.in(Compile).value.find{ a =>
64+
val info = a.get(moduleID.key)
65+
info.nonEmpty &&
66+
info.get.organization == "ch.epfl.lamp" &&
67+
info.get.name.startsWith("dotty-library")
68+
}
69+
if (dotttyLib.isEmpty) Def.task {
70+
streams.value.log.error("Dotty lib wasn't found")
71+
} else Def.task {
72+
run.in(Compile).toTask(s" -o output/stdLib -t ${dotttyLib.get.data} -d dotty-docs/docs -n dotty-lib -s library/src=https://github.com/lampepfl/dotty/tree/master/library/src#L").value
73+
}
74+
}.value
75+
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../README.md

scala3doc/documentation/index.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Scala3doc
2+
3+
**Documentation tool for Scala 3**
4+
5+
We are using [TASTy](https://github.com/lampepfl/dotty/blob/master/tasty/src/dotty/tools/tasty/TastyFormat.scala) to generate documentation. We aim to has all known and loved feature from scaladoc as well as new feature such as :
6+
7+
- integrated documentation and API
8+
- has option for basic pluggablity
9+
- and much more
10+
11+
**Yes, this page was generated using scala3doc**
12+
13+
You can learn more from out [documentation](scala3doc/index.html).
14+
15+
## Getting started
16+
17+
For now the recommended way to try out our project would be:
18+
- Clone our [repository](https://github.com/lampepfl/scala3doc)
19+
- Run `sbt main -n <name> -o <output> -t <tasty-files> -cp <classpath> -s { <sources> } ` where
20+
- `<name>`: name of module in generated documentation
21+
- `<output>`: location where documentation should be created
22+
- `<tasty-files>`: is list of dirs or jars that contains tasty files that should be documented
23+
- `<classpath>`: classpath that was used to generate tasty files
24+
- `<sources>`: links to source files of module that are used to link symbols on pages to their source file. They need to be supplied in form:
25+
local_dir=remote_dir#line_suffix e.g. src/main/scala=https://github.com/lampepfl/scala3doc/tree/master/src/main/scala#L
26+
27+
We also support `-d <documentation>` argument to provide static documentation. You can find more about that feature [here](static-page.html).

0 commit comments

Comments
 (0)