Time Nick Message 00:33 wsor4035 rubenwardy: you down if i just quick change the licenses to spdx? https://github.com/luanti-org/luanti-org.github.io/issues/394#issuecomment-3592058531 00:34 rubenwardy Idm 00:34 rubenwardy Seems fine 00:39 MTDiscord thanks 21:19 MTDiscord Spent some time learning more about remark and manually editing the POC (first draft was AI-generated 😬 ), hasn't been too difficult and remark is very feature-rich and well-documented. I'm happy with my plan: 1. doc-writers write Markdown in the "core namespace reference" section following our stricter "schema" or style guide to include arg types, return types, etc. in a strict format 2. remark parses Markdown to validate that 21:19 MTDiscord it's in the correct schema - if it's in the wrong schema, we send a clear error message (via GitHub Actions + PR status check + comment on the PR) before PR is merged 3. others can easily follow the schema and transform core namespace reference docs into other formats (e.g. TypeScript docs) Here is the v2 POC: https://github.com/mark-wiemer/hello-hello/blob/b2e643259c736be0d1ca6ddd27db8802d3de2db7/packages/remark/lua_api_transformed.md For 21:19 MTDiscord the schema itself, I'm thinking of following what already exists in the Lua space, just removing the need for the --- at the beginning of each line to mark something as a doc comment. Everything in this markdown file is a doc comment! I am currently revisiting https://github.com/LuaLS/lua-language-server which uses EmmyLua and other tech and is still actively maintained, haven't yet found a schema definition but it's a start! 21:27 MTDiscord Hm, that index.ts is not pretty. I fear the amount of knowledge it will take to edit the output format may be more than we want 21:33 MTDiscord > Everything in this markdown file is a doc comment! Conflicts with > doc-writers write Markdown in the "core namespace reference" section" Assuming you mean "a doc comment in code". We want writers to be writing in a single place rather than looking all over the source code. This wouldnt be as big of a problem if the engine was organized, but its not. Having good code comments would also be great, but is technically secondary. 21:39 MTDiscord hmm, maybe I should be extremely clear. EmmyLua is for Lua code files, so it is meant for files that contain both code and comments. A certain type of comment it supports is the doc comment, which is a line that starts with --- instead of the usual -- for non-doc comments. So EmmyLua is built specifically to parse lines that start with ---. However, we don't want our doc-writers to have to write --- at the beginning of each line of 21:39 MTDiscord the Markdown file, because the Markdown file doesn't contain code, it just contains text (that's what I meant when I said "everything in this markdown file is a doc comment"). All the Markdown that doc writers are writing will be transformed into doc comments, so they don't need to manually add the --- at the beginning of each line, it'd be redundant and annoying 21:40 MTDiscord I know how EmmyLua works, and I dont necessarily think a code-first approach is appropriate 21:40 MTDiscord For now I am not focusing on the idea of "adding comments to the actual Lua or C++ code and then transforming that into docs" because, well, we're just not there as a project. Maybe eventually, maybe, but with that route I see issues with, well, having to start every line with --- 21:40 MTDiscord I am not sure what you mean here 21:40 MTDiscord Not relevant 21:40 MTDiscord When you say "everything in this markdown file is a doc comment", what is a "comment"? 21:41 MTDiscord Just that it's not code, it's text primarily meant for human consumption, not machine execution. Yes there may be sample code snippets but even those are provided for human reference mainly 21:41 MTDiscord It was an imprecise usage for sure 21:42 MTDiscord When you put "emmylua" and "comment" in the same pargraph, I read "doc comment" as "comment in code that turns into docs" 21:42 MTDiscord (since that is how emmylua works) 21:42 MTDiscord I want to model the Markdown schema after EmmyLua so it's something familiar that works with Markdown instead of in-code comments 21:43 MTDiscord Understood. Which file there is the source from which the final md is generated? 21:45 MTDiscord I will ultimately want to update lua_api.md to follow the Markdown schema. I want to start with the "core namespace reference" section of that file which is currently about 2200 lines of Markdown. For now I've extracted that section to lua_api_out.md and I'm transforming that into lua_api_transformed.md. From now on though, I'll rename lua_api_out.md to core_namespace_ref.md and lua_api_transformed.md to 21:45 MTDiscord core_namespace_ref_transformed.md 21:46 MTDiscord Ah, right, my bad. I forgot we are just going with relatively stock MD 21:47 MTDiscord That's my plan at least 21:47 MTDiscord At some point there will be a source -> output pipeline to handle whatever basic transformations we need 21:48 MTDiscord I am unclear if there is a POC schema written yet or not 21:48 MTDiscord And to address this, yes, it's a very dirty POC. I promise I can clean it up a ton, add docs and tests, etc. Honestly, I'll probably be maintaining this as an unofficial project if it's not "approved" 🙂 21:49 MTDiscord When I said that I was under the assumption the index.ts was the schema, which I am unsure of now 21:49 MTDiscord no POC schema yet. Just wanted to give an update as I mentioned I'd be working on stuff this week. Thanksgiving was more taxing than expected but schema drafting is next. Going to be reading up on LuaLS and similar projects to find something maintained and useful 🙂 21:49 MTDiscord index.ts is the core of what will enforce the schema. It doesn't have any docs about the schema right now and is a very dirty, very early POC. No docs written anywhere yet, just toying around to learn remark at this point, now exploring, should have a draft schema within 7 days 😄 21:51 MTDiscord Here is the latest renamed file for reference, includes at least one bugfix from the past half hour: https://github.com/mark-wiemer/hello-hello/blob/3a7ec587e27b2bcd468712f080849c86a0bc40f7/packages/remark/core_namespace_ref_transformed.md 21:53 MTDiscord Notes for later: * Technically Markdown supports html comments, but it would be nice to have simpler comment options. Comments may be helpful for writers in many cases, but should be stripped out by the preprocessor * We should probably have an informal meeting about format/schema requirements * The format/schema needs to accommodate many levels of readers * Navigation needs a lot more focus than it has in the monolithic md doc 21:56 MTDiscord Consider my above message the v3 POC 🙂 22:07 MTDiscord Updated https://github.com/luanti-org/docs.luanti.org/issues/296 with this: "Once the API docs (or, at least the core namespace reference) are programmatically parseable, we can focus on providing a Language Server plugin for something like LuaLS so that modders can get insights while editing in an IDE instead of waiting to try to run their mod. However, this hasn't been discussed so I won't create a new issue for it." 22:07 MTDiscord ^ issue title is "Make API programmatically parseable" for IRC folks 22:56 MTDiscord https://github.com/LuaLS/LLS-Addons/pull/288/files integrates https://github.com/Voxrame/luanti-ide-helper into LuaLS, so we can definitely reference this (MIT-licensed) 🙂 23:01 MTDiscord https://github.com/luanti-org/docs.luanti.org/issues/296#issuecomment-3593759313 includes my current thoughts on what we should prioritize and how we can get this done 🙂 23:03 MTDiscord Please remember that "core namespace reference" is not holistic 23:04 MTDiscord Im not sure why you are singling it out so much on long term plans, when said plans definitely need to include the API reference as a whole 23:06 MTDiscord That's a good point, thoughts are still very flexible. I did skim through the rest of lua_api.md and there is of course a ton of critical info there. But I think we can agree that each big section currently has its own format for info, but also its own level of provided info. But yes, we should define a Markdown schema (or some other consistent format) that works for all sections of the API, not just core namespace reference 23:07 MTDiscord Each section having its own foramt and level of info is irrelevant to our plans since we want the whole thing to be consistent 23:08 MTDiscord Agreed, I think the "level of info" part that I said is also irrelevant 23:10 MTDiscord Also bear in mind that there are a few sections in the current docs site that should be integrated with the final API reference 23:11 MTDiscord (More than a few, I guess-- mostly a bunch of stuff under 'For Creators' 23:12 MTDiscord Hmm, yeah. My thoughts have mostly been about the schema, the tools to enforce the schema, and then publishing the schema. I think I've outlined the schema pretty well, next step is defining it, and yeah I should consider everything when writing it. It'll probably be longer than 7 days but I'll chip away at this as I have time and energy! 23:13 MTDiscord That thought was more relevant to your longer term plans about content and whatnot 23:13 MTDiscord In fact a lot of that lies outside of this immediate project, falling more under the "how do we organize the docs site" side of things 23:15 MTDiscord As for schema: If it were me, I would collect a lot of references for how big projects write their docs. Specifically projects with diverse user bases. Markdown and non-markdown. 23:16 MTDiscord Unfortunately a lot of projects are much better organized and section their APIs like a sane project 23:20 MTDiscord For instance, the LOVE2D docs are pretty good. Great amount of info, could have better navigation. Roblox may have an even worse problem with navigation. Godot looks nice but is also painful to navigate. Anything rustdoc has plenty of info but is somehow even worse to navigate. Phaser is ... kind of ok actually. I think a giant priority I have for docs is navigation. Detailed info is only useful if one can find it easily. 23:24 MTDiscord One of the specific pain points I think I often find is drilling down to the specific method I need. LOVE2D does well because it categorizes and alphabetizes large method lists and then you just navigate to that function for specifics. Godot particularly sucks because I have to drill through a sidebar to find anything. Roblox fails to categorize things cleanly. Rustdocs just ... doesnt present anything up-front for some reason. 23:26 MTDiscord lua_api.md only works well for a lot of us because we know what we are looking for most of the time and can search for keywords. For a newcomer, they dont know what they are looking for and the reference provides no overviews or categories 23:31 MTDiscord luals points to emmylua before the complete rewrite. nowadays emmylua refers to https://github.com/EmmyLuaLs/emmylua-analyzer-rust which has a slightly different flavour of annotation, but a slightly more powerful (and imo, more complete) type system. as for https://github.com/Voxrame/luanti-ide-helper, it still has yet to annotate any API above 5.8 and i believe they just simply use a lua_api.md from 5.8. this means it misses 23:31 MTDiscord stuff like "new" bone overrides. just something to keep in mind. 23:32 MTDiscord Ayy, didnt know EmmyLua got Rusty. 23:32 MTDiscord much, much faster too 23:32 MTDiscord processed voxelibre in a couple seconds 23:33 MTDiscord well, at least on my laptop 23:33 MTDiscord I will definitely be integrating luanti-tools with EmmyLua 23:35 MTDiscord it doesnt't fully reimplement luals tho. for instance, it doesn't have this addon+plugin system luals has. you have to obtain the definitions on your own and point to it with workspace.library key in .emmyrc.json 23:35 MTDiscord (simply dropping that into your workspace will work too, but i think that's a bit messy /shrug) 23:36 MTDiscord Is that enough to not use it? 23:36 MTDiscord no, i've used a different luanti API annotations project just fine 23:37 MTDiscord imo, the addon+plugin system was entirely designed with vscode users in mind. you don't get much benefit as a neovim, emacs, etc user 23:38 MTDiscord The addon system for which? Emmy or LuaLS? 23:38 MTDiscord emmylua doesn't have an addon+plugin system. all of luals's addons are available here: https://github.com/LuaLS/LLS-Addons 23:39 MTDiscord so, one cool thing to have in luanti-tools vscode extension might be automatically fetching/placing the annotations into the workspace 23:40 MTDiscord since emmylua asks users to provide definitions manually 23:40 MTDiscord Got it 23:40 MTDiscord I am also hoping EmmyLua even has a way to integrate other extensions with it in general. Would be unfortunate if not 23:42 MTDiscord If VSCode lets me talk to the server via LSP, then I will assume yes 23:46 MTDiscord @a.corp.serot are you saying LuaLS addons only work with VS Code? If so that's a bummer, I definitely want our solution to work across IDEs 23:46 MTDiscord ehh... emmylua is mostly a one-man/two-man work started at the end of 2024. very young compared to luals. when i started trying it at the start of this year it was very rough and unuseable. but i couldn't deny the performance it has compared to lua-written luals 23:47 MTDiscord kinda, but since addons are simply a schema to distribute definitions, it's trivial to manually obtain it and point luals to use it from neovim. 23:47 MTDiscord As a regular user of LuaLS ... I wouldnt say it is perfectly stable either ;p 23:48 MTDiscord Yeah I never bothered asking what people currently use, I just wrote my own experience. What do you guys use? 23:48 MTDiscord Mind, our core "solution" is consistent source which can be easily converted to any format needed. It should be "trivial" to generate both LuaLS and EmmyLua definitions. 23:49 MTDiscord vscode with a neovim subprocess which means i get to configure two editors lol 23:49 MTDiscord Yes, the proposed solution should include that 23:49 MTDiscord what does the neovim subprocess do? 23:49 MTDiscord In fact, remark should be able to provide a very light JS interface that outputs components of the parsed tree for other programs to take advantage of 23:50 MTDiscord makes vscode act more like neovim. i can use most neovim features from within vscode 23:50 MTDiscord gets me a large coverage of things i can use, since i can use both neovim and vscode stuff 23:51 MTDiscord For example, I should be able to iterate through all objects/methods and retrieve their names, types, outputs, etc. and then write that however I need to, like for LuaLS and EmmyLua definitions. 23:51 MTDiscord yes, this is why I chose it 🙂 23:52 MTDiscord What Luanti-specific extensions do you use, @a.corp.serot and/or @greenxenith ? 23:52 MTDiscord Like I said, I use LuaLS at the moment 23:52 MTDiscord (sumneko.lua) 23:52 MTDiscord just emmylua, since luanti-tools kinda don't have much use to me. how are you gonna define types mark? 23:53 MTDiscord Types get defined in documentation 23:53 MTDiscord OK, just wondering if there were other extensions like luacheck or LuaLS add-ons like https://github.com/LuaLS/LLS-Addons/tree/main/addons/luanti-full-api or https://github.com/LuaLS/LLS-Addons/tree/main/addons/luanti. Any config details are helpful 23:53 MTDiscord (at some point in the distant future, types should probably be defined in-code and then extracted into the docs, but that is a problem for future us) 23:54 MTDiscord I dont think that LuaLS luanti addon even exists anymore 23:54 MTDiscord I'm on the fence about this, but agreed that it's not a conversation for right now. Focus for right now is a consistent doc format that can ultimately enable cross-IDE API support 23:54 MTDiscord lua_api.md doesn't have all types necessary to express the luanti API... unless you don't care about type accuracy and just opaquely do stuff like "this is a table with any stuff in it" 23:54 MTDiscord But those links exist? I am still learning LuaLS 23:54 MTDiscord Hence our goal to drastically improve it! 23:54 MTDiscord The module in https://github.com/LuaLS/LLS-Addons/tree/main/addons/luanti does not exist 23:55 MTDiscord The module in https://github.com/LuaLS/LLS-Addons/tree/main/addons/luanti-full-api links to the Voxrame ide helper 23:55 MTDiscord oh right, that one's fgaz's isn't it 23:55 MTDiscord We can make iterative improvements once we have a consistent schema, ref https://github.com/luanti-org/docs.luanti.org/issues/296#issuecomment-3593759313 23:55 MTDiscord https://codeberg.org/fgaz/luanti-lls-definitions 23:55 MTDiscord yes, I was just including it as a sample one 23:56 MTDiscord fgaz's is very incomplete, so you don't really need to look into it 23:56 MTDiscord A while ago luacheck was a dependency of the LuaLS extension 23:56 MTDiscord I dont know if that is still the case, but I believe I have both installed 23:56 MTDiscord Not that it matters 23:56 MTDiscord good to know, as you can tell I'm very early in research and haven't written hello world yet with some of these tools 🙂 23:57 MTDiscord you don't need luacheck for luals these days (i never had one myself) 23:57 MTDiscord Anyway, the language server is sort of irrelevant 23:57 MTDiscord As long as you can extract the fundamental info every LS needs and every (competant) API ref provides, we are good 23:58 MTDiscord That is the goal! 🙂 23:59 MTDiscord kinda depends on how expressive the type system you want. the annotations i wrote (and abandoned) struggled with trying to be compatible with both language server. the annotations andro wrote struggled with how inexpressive the type system in both language servers are.