| Time |
Nick |
Message |
| 03:40 |
MTDiscord |
<mark.wiemer> Hi everyone, I'm back π my new project will be the Luanti TypeScript API, and I'm curious for thoughts on how the core team would feel about changes to the lua_api.md file to make it more consistently parseable. The workflow I imagine is: 1. Parse lua_api.md into an AST 1. Extract the core namespace reference section 1. Identify all the functions, their arguments, and their descriptions 1. Generate a TypeScript API from that |
| 03:40 |
MTDiscord |
Obviously, the actual types of the arguments is going to be a bit more complicated. Right now there isn't a consistent way to programmatically determine the type of each argument, some are documented and some are implied, e.g. "if x is set to true, then ..." means x is a boolean, but it's never explicitly stated Β I'm thinking of adding some consistent typing to the docs, but of course Lua doesn't have that and I want to propose a change that the |
| 03:40 |
MTDiscord |
team is willing to accept. Some ideas are simple: change core.mvdir(source, destination) to core.mvdir(source: Vector3, destination: Vector3) but that might be too weird for Lua. There's also Β core.mvdir(source, destination) - source (Vector3): the start location - destination (Vector3): the end location Β But this encourages https://blog.codinghorror.com/avoiding-undocumentation/ which nobody likes. Β Thoughts? Ideas? Β Hope you've all been well |
| 03:40 |
MTDiscord |
π |
| 03:46 |
|
mrcheese joined #luanti-docs |
| 03:50 |
MTDiscord |
<wsor4035> > Luanti TypeScript API |
| 03:50 |
MTDiscord |
<wsor4035> https://tenor.com/view/dj-khaled-another-one-one-gif-5057861 |
| 03:52 |
MTDiscord |
<wsor4035> sounds like https://github.com/luanti-org/luanti/issues/13926 is sorta what your looking for |
| 05:00 |
|
MTDiscord joined #luanti-docs |
| 05:09 |
MTDiscord |
<mark.wiemer> Nah I want TS, Lua has so little tooling available. Formatters, linters, types, package managers, all much better in TS from what I've seen. I only know of one other TS API and it's very incomplete. And I like to think I have a good track record of actually getting things done π |
| 05:11 |
MTDiscord |
<wsor4035> your missing the point. you want docs that can be used programtically, lua headers would (at least somewhat) |
| 05:11 |
MTDiscord |
<mark.wiemer> Yes agreed, I know that much π |
| 05:11 |
MTDiscord |
<wsor4035> also there is like 4 typescript projects |
| 05:11 |
MTDiscord |
<mark.wiemer> Ah, happy to follow links, I only found the one that was reasonable |
| 05:13 |
MTDiscord |
<wsor4035> there is the one by whatever there face is that is in a tower defense like game, supports jsx to formspecs. iirc not open source there is one that whoever made + then forked off by fgax |
| 05:13 |
MTDiscord |
<wsor4035> https://leanrada.com/wares/miniforts/ first one is buried in here |
| 05:13 |
MTDiscord |
<mark.wiemer> Haha not much to follow there, closed source and "a fork of fgax" isn't very encouraging. Surely you agree more tooling would be better? |
| 05:14 |
MTDiscord |
<wsor4035> mostly just dumping the ones i can think of off the top of my head |
| 05:14 |
MTDiscord |
<wsor4035> https://github.com/RepComm/mt-api |
| 05:17 |
MTDiscord |
<mark.wiemer> Doesn't look like mini forts has been updated in 4 years, might be something there tho :/ |
| 05:18 |
MTDiscord |
<wsor4035> https://github.com/jordan4ibanez/crafter/blob/23def382ab8acd30e8bdd840705439dda1d894fc/minetest-api.d.ts |
| 05:20 |
MTDiscord |
<wsor4035> anyways, handfuls of them around, none has ever "won" |
| 05:20 |
MTDiscord |
<mark.wiemer> My project is a fork of mt-api yes |
| 05:21 |
MTDiscord |
<mark.wiemer> I'm built different π there were lots of doc website proposals around for a while too! |
| 05:23 |
MTDiscord |
<wsor4035> ....im just not going to engage on that |
| 05:24 |
MTDiscord |
<wsor4035> anyways, back to machine readable/parsable docs, should annoy green, they want similar. one of there βΎοΈ projects that will be done in 2069 |
| 05:25 |
MTDiscord |
<mark.wiemer> Sorry, I didn't meant to take credit for others' work. But I helped, didn't I? We got a lot done late last year, a lot of stuff that folks said wouldn't happen. I think the negativity around a TS API is very similar. |
| 05:26 |
MTDiscord |
<mark.wiemer> Crafter looks pretty good, lots of updates in the past six months π |
| 05:26 |
MTDiscord |
<wsor4035> maybe im coming off as negative, but just saying there has been a handful of them before that may be useful for you to steal borrow π from |
| 05:27 |
MTDiscord |
<mark.wiemer> Fair enough, appreciate the links π |
| 05:27 |
MTDiscord |
<wsor4035> i dont really give a shit what you program in, more so am just annoyed when a project that uses typescript doesnt have good lua api docs to use it |
| 05:28 |
MTDiscord |
<wsor4035> since you cant just read self documenting code |
| 05:28 |
MTDiscord |
<mark.wiemer> I'm confused if you're annoyed at my proposal or at the core devs for not documenting their code programmatically |
| 05:29 |
MTDiscord |
<wsor4035> im annoyed at people who make mods/games with typescript and dont provide lua api docs |
| 05:29 |
MTDiscord |
<mark.wiemer> Fair enough, I plan on contributing back upstream for sure, hence my original proposals π |
| 05:30 |
MTDiscord |
<wsor4035> mt annoyance is unrelated to your goal here. as to your goal seems legit as long as it can be useful to others and doesnt become a pain/maintaince burden |
| 05:30 |
MTDiscord |
<mark.wiemer> Sounds good π |
| 05:31 |
MTDiscord |
<wsor4035> (also repcom is on matrix if you want to see if they are still around) |
| 05:31 |
MTDiscord |
<mark.wiemer> Might be worth trying to set that up π |
| 05:32 |
MTDiscord |
<greenxenith> > I'm curious for thoughts on how the core team would feel about changes to the lua_api.md file to make it more consistently parseable Did you forget the projects you were (are?) a part of? A major part of organizing the API team was to convert lua_api.md to a machine- and human-readable format |
| 05:33 |
MTDiscord |
<mark.wiemer> I have forgotten, I suppose π |
| 05:33 |
MTDiscord |
<greenxenith> Like, thats the main reason I barely maintain luanti-tools is id rather focus on making the API parseable to generate official API snippets |
| 05:33 |
MTDiscord |
<greenxenith> When I have time to dedicate, that is |
| 05:34 |
MTDiscord |
<wsor4035> topological things getting pushed off the top after jam? π |
| 05:34 |
MTDiscord |
<greenxenith> In fact, the API team exists because Benrob, Luatic, and I wanted the lua_api to be parseable |
| 05:34 |
MTDiscord |
<wsor4035> that should be vague enough |
| 05:35 |
MTDiscord |
<mark.wiemer> Well, let's just say that's my goal now too |
| 05:35 |
MTDiscord |
<greenxenith> It got sidelined because we were able to gain more traction moving the wikis into a unified doc page, but most of the short term goals are complete by now. That original goal is still in the roadmap for the docs project |
| 05:36 |
MTDiscord |
<greenxenith> https://github.com/luanti-org/docs.luanti.org/issues/47 |
| 05:36 |
MTDiscord |
<greenxenith> (you actually reacted to comments this issue, mark ;p ) |
| 05:36 |
MTDiscord |
<mark.wiemer> Hey it's been a long year! |
| 05:37 |
MTDiscord |
<greenxenith> The original project was to convert the monolith file to AsciiDoc, but I dont think that was the correct choice now that we tried it in an extended capacity |
| 05:37 |
MTDiscord |
<wsor4035> asciidoc can burn in hell |
| 05:37 |
MTDiscord |
<greenxenith> AsciiDoc, while , is very cumbersome to write. |
| 05:37 |
MTDiscord |
<mark.wiemer> I think Markdown with an automated check system should be sufficient, I've been using remark and it's really good |
| 05:37 |
MTDiscord |
<greenxenith> while parseable* |
| 05:38 |
MTDiscord |
<greenxenith> The problem is markdown is missing a lot of very useful elements |
| 05:38 |
MTDiscord |
<mark.wiemer> Like what? Anything that couldn't be handled by an external parser/checker? |
| 05:39 |
MTDiscord |
<greenxenith> I think it was mostly admonitions, includes, and macros |
| 05:39 |
MTDiscord |
<greenxenith> I had a proposed alternative and I dont remember what it was |
| 05:40 |
MTDiscord |
<mark.wiemer> MDX is close, but I'm thinking a simple JS script could expand macros, etc |
| 05:40 |
MTDiscord |
<wsor4035> admonitions is solved now in markdown |
| 05:40 |
MTDiscord |
<greenxenith> I think it was one of the markdown flavors with admonitions + a preprocessor |
| 05:40 |
MTDiscord |
<mark.wiemer> Yeah I like preprocessors nowadays π |
| 05:41 |
MTDiscord |
<greenxenith> Pandoc |
| 05:41 |
MTDiscord |
<greenxenith> https://discord.com/channels/369122544273588224/926231483155378176/1300317894848020501 |
| 05:42 |
MTDiscord |
<greenxenith> Not mark joining literally 3 days later https://discord.com/channels/369122544273588224/926231483155378176/1301755758047662132 |
| 05:42 |
MTDiscord |
<mark.wiemer> I've been rethinking Pandoc, but I never gave it a fair shot |
| 05:43 |
MTDiscord |
<greenxenith> I briefly tested it |
| 05:43 |
MTDiscord |
<greenxenith> I am not attached to any particular tool |
| 05:43 |
MTDiscord |
<greenxenith> Pandoc just had a lot of advantages |
| 05:43 |
MTDiscord |
<greenxenith> Two major ones being: We dont have to maintain it ourselves, and it supports Lua macros |
| 05:44 |
MTDiscord |
<mark.wiemer> How useful are Lua macros? |
| 05:44 |
MTDiscord |
<greenxenith> Whether or not you like Lua as a tooling language, it means we can build snippets using what Luanti comes bundled with instead of, say, an entire JS engine |
| 05:44 |
MTDiscord |
<greenxenith> "Lua macros" = you can script pandoc to preprocess your markdown using Lua |
| 05:45 |
MTDiscord |
<greenxenith> Specifically I think I was writing a {{mustache}} macro replacer for pandoc |
| 05:45 |
MTDiscord |
<wsor4035> except you need pandoc which is external anyways rendering your point moot |
| 05:45 |
MTDiscord |
<mark.wiemer> I don't see significant downside in onboarding JS. Yes it's a new lang, but I imagine it'd have a lot more tools than Lua |
| 05:46 |
MTDiscord |
<greenxenith> This is true |
| 05:46 |
MTDiscord |
<wsor4035> something like ldoc would be more inline with your comments |
| 05:46 |
MTDiscord |
<wsor4035> (if we added it in) |
| 05:46 |
MTDiscord |
<greenxenith> That is correct, and I very much do not like ldoc |
| 05:46 |
MTDiscord |
<greenxenith> We tried ldoc |
| 05:46 |
MTDiscord |
<greenxenith> it was painful |
| 05:46 |
MTDiscord |
<mark.wiemer> I think step one is identifying what features we need, then we can browse tools |
| 05:46 |
MTDiscord |
<wsor4035> the group never did, you might have |
| 05:46 |
MTDiscord |
<greenxenith> Done and done |
| 05:47 |
MTDiscord |
<mark.wiemer> "done and done" to what? |
| 05:47 |
MTDiscord |
<wsor4035> at least iirc anyways |
| 05:47 |
MTDiscord |
<greenxenith> "We" meaning a couple of us have used ldoc |
| 05:47 |
MTDiscord |
<greenxenith> Luatic, myself, and I believe Ben |
| 05:47 |
MTDiscord |
<wsor4035> anyways, personally i would pick mdx over pandoc these days |
| 05:49 |
MTDiscord |
<greenxenith> We need a markdown preprocessor/templating engine that ideally can export to an easier to read data format (yes you can strictly format and read markdown, but it would be better if I could export to XML or JSON or something) |
| 05:49 |
MTDiscord |
<greenxenith> Rendering the markdown is a separate issue that is basically already solved |
| 05:49 |
MTDiscord |
<mark.wiemer> remark turns Markdown into an AST |
| 05:51 |
MTDiscord |
<mark.wiemer> Then we can turn that into whatever, of course. The issue is that we'll need a more strict guideline than "syntactically valid Markdown", we'll need a description of what properties need to be documented (args, types, descriptions) and how they should be syntactically structured for consistent parsing |
| 05:51 |
MTDiscord |
<greenxenith> And that is the next issue, which has also kind of been tackled already |
| 05:51 |
MTDiscord |
<wsor4035> can enforce that via ci |
| 05:51 |
MTDiscord |
<greenxenith> Should enforce during editing |
| 05:51 |
MTDiscord |
<mark.wiemer> Yep π |
| 05:52 |
MTDiscord |
<wsor4035> personally i think its better to keep stuff in markdown everyone understands rather than raising a barrier of understand whatever tooling one feels like |
| 05:52 |
MTDiscord |
<wsor4035> speaking of ci, do need to fix cspell |
| 05:52 |
MTDiscord |
<mark.wiemer> Also possible π I want to focus on critical requirements first, CI is critical, editing less so |
| 05:52 |
MTDiscord |
<greenxenith> This is in response to? |
| 05:52 |
MTDiscord |
<wsor4035> in general to both of you |
| 05:52 |
MTDiscord |
<greenxenith> I didnt think anyone disagreed that we wanted markdown |
| 05:53 |
MTDiscord |
<mark.wiemer> Never hurts to clarify π |
| 05:53 |
MTDiscord |
<greenxenith> But it does hurt to move one if someone thinks the intent is something else |
| 05:53 |
MTDiscord |
<wsor4035> more so why do some flavor of imports, other rules, etc when you can just have the standard in a format and use remark for example to do the transforms |
| 05:53 |
MTDiscord |
<greenxenith> move on* |
| 05:54 |
MTDiscord |
<greenxenith> We will still absolutely have templating ("imports") |
| 05:54 |
MTDiscord |
<greenxenith> Thats what remark is for |
| 05:54 |
MTDiscord |
<wsor4035> why tho? make a markdown schema and roll with it |
| 05:54 |
MTDiscord |
<greenxenith> What? We need to be able to re-use content consistently |
| 05:55 |
MTDiscord |
<greenxenith> Like type annotation. All types should backlink, and we are NOT manually filling those out |
| 05:55 |
MTDiscord |
<wsor4035> remark can export markdown to an ast. if its in a specified schema then they will all be the same |
| 05:55 |
MTDiscord |
<greenxenith> Im not even talking about exporting |
| 05:55 |
MTDiscord |
<wsor4035> at least running with what mark said, havent looked at remark in depth so could be full of shit |
| 05:55 |
MTDiscord |
<greenxenith> I am talking about in the documentation formatting |
| 05:56 |
MTDiscord |
<greenxenith> Like I am not going to write Type: Position or whatever every time I annotate a Pos type. That is going to break if the section changes or someone is going to forget one. Sure you could enforce via formatting but why put the effort into the enforcement if you could template it for cheaper? |
| 05:58 |
MTDiscord |
<greenxenith> The same concept goes for version tagging, deprecation notices, page headers, etc |
| 05:58 |
MTDiscord |
<greenxenith> I am not saying we need anything crazy. Just the ability to template the markdown. Which remark should provide? |
| 05:59 |
MTDiscord |
<greenxenith> Templating was the one feature from ADoc that was genuinely important |
| 06:01 |
MTDiscord |
<greenxenith> If there is a Markdown flavor that already does this, great, but I wasnt aware of any mainstream ones |
| 06:02 |
MTDiscord |
<greenxenith> I mean I guess MDX technically, but thats cheating |
| 06:05 |
MTDiscord |
<greenxenith> To wsor's point, better to keep as close as possible to commonmark, with the necessary feature to stay as organized and stable as possible |
| 06:05 |
MTDiscord |
<greenxenith> Putting JS inside a markdown file feels wrong |
| 06:06 |
MTDiscord |
<mark.wiemer> remark can take any Markdown and turn it into anything else. If we have a strict schema we don't have to handwrite back links, we can write a remark plugin to recognize where a back link should be added. So it's both a parser and a templating engine in that sense yes |
| 06:07 |
MTDiscord |
<greenxenith> Does recognizing where the link should be added equate to adding it? I dont want to write it myself |
| 06:08 |
MTDiscord |
<mark.wiemer> Yes, we write a plugin to say where a link should be, and in that same plugin write the template expansion for the links. Nobody would have to handwrite any back links |
| 06:09 |
MTDiscord |
<mark.wiemer> I will work on some schema proposals with sample input output for reference |
| 06:10 |
MTDiscord |
<greenxenith> There are also some things that need to be wrapped in consistent elements. For examples, parameters often get thrown in tables that have a type, value, and note column. Tables are a pain to write usually, so converting a simple comma-separated list (likely similar to how it would be written in code) would get converted to a table |
| 06:10 |
MTDiscord |
<mark.wiemer> Yes, we should have the schema enforce all that π |
| 06:10 |
MTDiscord |
<greenxenith> Sounds good to me then |
| 06:11 |
MTDiscord |
<greenxenith> (I was already satisfied with the solution, I am just here to defend my templates) |
| 06:11 |
MTDiscord |
<mark.wiemer> Yes, I see the value of templates now π |
| 06:11 |
MTDiscord |
<greenxenith> There is something to be said about documentation per-method/object vs trying to categorize them ourselves, but that can be bikeshed another day. |
| 06:12 |
MTDiscord |
<greenxenith> FWIW we should probably have a preliminary extractor that dumps as much function/object data as it can from the engine itself |
| 06:16 |
MTDiscord |
<greenxenith> And future note: I am sure there is some way to do optional schema or something, with which we might eventually have one which adds some flags for the HTML export to pretify some things (like icons, borders, colors, etc) |
| 06:17 |
MTDiscord |
<greenxenith> Either way, would be stoked if this works well. Have been wanting officially-generated data for language servers for a long time |
| 06:19 |
MTDiscord |
<greenxenith> This is also coming within a week of API autocomplete being brought up in a different channel |
| 06:21 |
MTDiscord |
<mark.wiemer> Created https://github.com/luanti-org/docs.luanti.org/issues/296: Make API programmatically parseable |
| 06:21 |
MTDiscord |
<greenxenith> Apologies for harsh tones, I didn't mean to push so much. Im just glad Mark isnt dead and we have actionable movement. |
| 06:23 |
MTDiscord |
<greenxenith> Another note from prior discussions: I dont actually remember what concensus was, but there was lengthy debate about how to balance developers writing documentation with their code (in-code or on the side) and writers writing/editing docs. Unfortunately probably not going to land a perfect solution |
| 06:24 |
MTDiscord |
<mark.wiemer> The important part here is for devs to see the value of good docs. If there is value, low friction, and quick automated enforcement, it shouldn't be too hard to push for |
| 06:24 |
MTDiscord |
<greenxenith> ... and I think you were part of that discussion because early on you were concerned about in-file code docs |
| 06:25 |
MTDiscord |
<mark.wiemer> Yeah I vaguely remember. Honestly it's been a very long year though!! Most important would be having discussions with the devs themselves to see how they feel, what concerns they have, etc |
| 06:26 |
MTDiscord |
<greenxenith> My 2 cents for now: Well-structured docs in-code requires well-structured code, which Luanti is not. I would avoid relying on engine structure to keep track of what needs documentation and where it belongs |
| 06:26 |
MTDiscord |
<mark.wiemer> My focus will definitely be "take lua_api.md and make it better" more than looking at engine internals |
| 06:28 |
MTDiscord |
<greenxenith> Their input is important, but keep in mind: Most features are written by people that arent the core devs. Whats important is a good workflow for the common developer + docs writers. We will end up doing most of the grunt work to convert anyway. |
| 06:32 |
MTDiscord |
<mark.wiemer> v1 of transforming current lua_api.md to TS docs, for anyone curious what remark can do for a noob: https://github.com/mark-wiemer/hello-hello/blob/main/packages/remark/lua_api_transformed.md |
| 06:47 |
MTDiscord |
<mark.wiemer> https://github.com/luanti-org/docs.luanti.org/issues/297 has my initial thoughts. Pretty late for me here and I'm on-call tmrw, then travelling for Thanksgiving. I'll keep chipping away at this though, the idea of making a Markdown parser, then a lil language server and a VS Code extension sounds fun π current work is defining a schema though |
| 14:14 |
MTDiscord |
<a.corp.serot> mark, how familiar are you with emmylua? the API spec should provide enough information to generate definition files for existing language servers. emmylua's the only one i think worth pursuing since luals development stalled. there's another idea that sfan5 proposed using yaml instead for less parsing work. |
| 14:34 |
MTDiscord |
<luatic> Any way to write down types and associated documentation works, doesn't have to be YAML. Personally I'd be gravitating towards a Lua DSL. In either case it seems to me we're rehashing a lot of the discussion in https://github.com/luanti-org/luanti/issues/13926 here. |
| 15:17 |
MTDiscord |
<wsor4035> its linked above π |
| 15:19 |
|
Desour joined #luanti-docs |
| 16:57 |
MTDiscord |
<mark.wiemer> Generally I want to avoid Lua-based tools just because of the very weak Lua ecosystem. Luarocks isn't a great package manager, code linters and formatters are weak, and overall the language just doesn't have a lot of usage compared to JavaScript. I'm happy to look at proofs-of-concept but just naming tools will result in a flood of options without clarity as to which ones can actually do the job we need. |
| 16:59 |
MTDiscord |
<mark.wiemer> I plan to read over that incident as well. Ultimately I think the need has been clear for some time, but a reasonable solution hasn't been written. We've spent time on tools but not on details or proofs-of-concept. A POC would push this work a lot further than proposing yet another tool without any further details. Again, very open to new tools if they are backed by a proof-of-concept |
| 17:00 |
MTDiscord |
<mark.wiemer> Lol I sent the message "s/incident/GitHub issue" and Discord actually edited the message! Sorry IRC folks lol |
| 17:23 |
rubenwardy |
23qwa |
| 18:22 |
MTDiscord |
<luatic> Well, as I've written in the use, LΓVR has demonstrated that a Lua-based DSL can work well for this kind of thing: > See e.g. LΓVR for an example of a custom Lua-based DSL format that works fine: https://github.com/bjornbytes/lovr-docs/blob/v0.18.0/api/lovr/math/Mat4/rotate.lua (rendered output: https://lovr.org/docs/Mat4:rotate). |
| 19:56 |
MTDiscord |
<greenxenith> It would probably be a good idea to leverage the existing standards for lua language servers. Reinventing the wheel is generally a bad idea. |
| 20:00 |
MTDiscord |
<greenxenith> That discussion should really be moved to the docs repo |
| 20:04 |
MTDiscord |
<greenxenith> Its nice to talk about options but we only really get anywhere by trying things. |
| 21:24 |
|
user333_ joined #luanti-docs |
| 21:33 |
|
Niklp joined #luanti-docs |
| 21:34 |
|
mrcheese joined #luanti-docs |
| 23:49 |
MTDiscord |
<wsor4035> should i just yolo it over and see who screams? |
| 23:49 |
MTDiscord |
<wsor4035> also why do we have two issues for this |
| 23:55 |
user333_ |
i did not expect #298 to get merged within 2 minutes of me opening it |
| 23:56 |
MTDiscord |
<wsor4035> literally the most trival pr ever |
| 23:56 |
user333_ |
yea |
| 23:56 |
user333_ |
1 letter changed |
| 23:56 |
user333_ |
added* |
