| Time |
Nick |
Message |
| 00:20 |
|
tacokoneko_ joined #luanti-dev |
| 01:09 |
[MatrxMT] |
<repetitivestrain> It would be useful if custom mapgens could implement get_biome_data and provide biome maps from Lua |
| 01:09 |
[MatrxMT] |
<repetitivestrain> Not necessarily in advance |
| 01:11 |
|
SFENCE joined #luanti-dev |
| 01:29 |
|
cbrpnk joined #luanti-dev |
| 02:44 |
|
SFENCE joined #luanti-dev |
| 02:46 |
|
aliasstilltaken joined #luanti-dev |
| 03:02 |
|
SFENCE joined #luanti-dev |
| 04:00 |
|
MTDiscord joined #luanti-dev |
| 04:38 |
|
fluxionary joined #luanti-dev |
| 04:54 |
|
SFENCE joined #luanti-dev |
| 05:14 |
MTDiscord |
<corarona> yeah true |
| 05:15 |
|
repetitivestrain joined #luanti-dev |
| 05:36 |
|
SpaceManiac joined #luanti-dev |
| 06:58 |
|
SFENCE joined #luanti-dev |
| 07:16 |
|
SFENCE joined #luanti-dev |
| 07:37 |
|
v-rob joined #luanti-dev |
| 08:04 |
|
sugarbeet joined #luanti-dev |
| 08:15 |
|
SFENCE joined #luanti-dev |
| 08:24 |
|
SFENCE joined #luanti-dev |
| 08:32 |
Krock |
turns out that mip mapping is disabled on RTTs. After enabling, the nametag scaling looks a lot better: https://i.postimg.cc/gjwbDzSG/out.png |
| 08:32 |
Krock |
this is about the best result that I could achieve so far |
| 08:32 |
|
SFENCE joined #luanti-dev |
| 08:48 |
|
Farooq joined #luanti-dev |
| 08:53 |
Farooq |
Hello. I just realized something and I wanted to talk about my concern. I realized now we've got core.find_path. I became both happy and worry. I am happy because in CTF, we needed such a thing. And this might be useful. But I'm worried because I think Luanti is going the wrong direction. I have no experience working on the engine development. But I thought the goal is dehardcoding things. But adding such these features goes right in the opposite |
| 08:53 |
Farooq |
direction IMO. I might be wrong. But a small engine which you can add modules and plugins to it IMO is much better than having a big engine with tons of features. Because usually each game uses only a small subset of these features. |
| 08:55 |
Farooq |
In the last 2 roadmap brainstorms, dehardcoding has been recommended multiple times in each. Also a feature to load dynamic modules has been suggested several times. |
| 08:55 |
Farooq |
For instance extending the Lua API with custom Rust/C/C++ modules or libraries. |
| 08:57 |
Farooq |
maybe this should be discussed in a Github issue. Let me know and I will open an issue or contribute to an already existing one |
| 09:00 |
Farooq |
As a reminder, many big and popular servers are already using their custom fork of Luanti |
| 09:02 |
Krock |
Farooq: custom modules are either impractical (needs compiling), or insecure (cannot easily be sandboxed to prevent malicious code) |
| 09:02 |
Farooq |
server admins are already using their custom fork |
| 09:02 |
Farooq |
some server owners |
| 09:03 |
repetitivestrain |
And? |
| 09:03 |
Krock |
yes. what's the problem with that? |
| 09:03 |
repetitivestrain |
if mine is any indication, i doubt they've diverged much |
| 09:03 |
repetitivestrain |
you can already extend minetest with C other native modules anyway by means of lua's standard module interface or the LuaJIT FFI |
| 09:03 |
repetitivestrain |
see here: https://codeberg.org/mineclonia/mineclonia/src/branch/mineclonia_mapgen/mods/MAPGEN/mcl_levelgen/ffi.c |
| 09:03 |
Farooq |
not anything fits within Lua |
| 09:03 |
Krock |
Look at the Wine project. I don't think there's many people who use the official builds, and instead add patches to improve compatibility with certain games. |
| 09:05 |
Krock |
the goal of Luanti should be the same: to provide a generic API that fits well enough to cover most use-cases. If server owners still need more, they are free to suggest certain API, implement them in Lua, or use the insecure environment |
| 09:05 |
Krock |
to import a C/C++ library with Lua exports |
| 09:05 |
Farooq |
repeating myself, not anything fits as a mod |
| 09:05 |
Farooq |
otherwise server owners wouldn't go for patching the engine |
| 09:05 |
repetitivestrain |
well to take your example of core.find_path, mineclonia has had a very efficacious lua pathfinder for nearly half a year |
| 09:06 |
Krock |
Farooq: http://lua-users.org/wiki/BuildingModules |
| 09:06 |
repetitivestrain |
and there's been absolutely no need to patch the engine |
| 09:06 |
Krock |
alternative link: https://piembsystech.com/integrating-lua-modules-into-c-programs/ |
| 09:06 |
Krock |
oops scratch the 2nd one. it's the wrong direction. |
| 09:08 |
Krock |
code/library example for a C library exposed to Lua, which can be used within Luanti: https://github.com/lunarmodules/luasocket |
| 09:09 |
Krock |
repetitivestrain: I regularly patch it to get the 3D buttons back |
| 09:09 |
repetitivestrain |
yes and i've a series of patches that implement apis enabling CSMs to override client object and local player physics |
| 09:09 |
Krock |
and some servers tweak their anticheat code for various reasons |
| 09:10 |
repetitivestrain |
which are used by this quasi-official mineclonia CSM: https://codeberg.org/halon/mcl_localplayer/ |
| 09:10 |
Farooq |
Krock, repeating myself for the third time, not anything fits as a Lua mod for Luanti. I know all of them are possible. But there are certain cases in which the server owners need to patch the engine |
| 09:10 |
Krock |
Farooq: so what? |
| 09:11 |
repetitivestrain |
generally in my experience it's the client that must be patched |
| 09:11 |
Krock |
repetitivestrain: hehe I've also got a set of patches ready to test mods against vulnerabilities :3 |
| 09:12 |
Farooq |
Krock, In light of my idealism, the engine shouldn't need to be patched at all. If that "ideal" is not possible. I think Luanti should go for the way which requires least number of patches |
| 09:12 |
repetitivestrain |
Farooq: what are your proposals? |
| 09:12 |
repetitivestrain |
why don't you submit them individually |
| 09:13 |
Farooq |
wdym? |
| 09:13 |
repetitivestrain |
there is not and cannot be a one size fits all C++ API, you know |
| 09:13 |
|
SFENCE joined #luanti-dev |
| 09:13 |
Farooq |
I don't follow. What doy you mean by all C++ API? |
| 09:13 |
Farooq |
s/doy/do |
| 09:14 |
[MatrxMT] |
<birdlover32767> 100 prs on the luanti repo yet again |
| 09:14 |
repetitivestrain |
you were proposing to introduce facilities for loading native modules, but the material question is what APIs do you need which are not currently available to Lua |
| 09:14 |
Krock |
birdlover32767: review them 8) |
| 09:14 |
Farooq |
repetitivestrain, nope that's only half of my proposal |
| 09:14 |
repetitivestrain |
What's the rest |
| 09:14 |
Farooq |
the other half is that the engine should provide the least |
| 09:15 |
Farooq |
that is there shouldn't be core.find_path |
| 09:15 |
Farooq |
unless it is loaded by a dynamic module on demands of server owners |
| 09:15 |
repetitivestrain |
How does core.find_path interfere with your (very general) proposal |
| 09:15 |
repetitivestrain |
you are under no obligation to use it and serious Luanti mods do not exercise the built-in pathfinder anyway |
| 09:15 |
Farooq |
my proposal is a very small engine which can be extended to whatever you want using native or non native modules |
| 09:16 |
repetitivestrain |
and how precisely would you set about realizing such an engine |
| 09:16 |
repetitivestrain |
the devil is in the details |
| 09:16 |
Krock |
ABI compatibility is annoying to deal with |
| 09:16 |
repetitivestrain |
what apis would you expose to lua and native modules? |
| 09:17 |
Farooq |
repetitivestrain, it seems I need to really write a detailed proposal which doesn't fit in chat |
| 09:18 |
Farooq |
for example there shouldn't be APIs in core namespace about "trees" and "ores" |
| 09:18 |
Farooq |
I understand my idea is not yet well formed. But instead of immediately rejecting my idea, please iterate over it and improve it |
| 09:18 |
repetitivestrain |
look i've just one question, how would you render the engine sufficiently extensible for these prospective modules to implement https://github.com/luanti-org/luanti/pull/16429 |
| 09:19 |
repetitivestrain |
which is the only reason why i continue to patch my server |
| 09:19 |
Farooq |
lemme open it. I don't have any experience with Graphics |
| 09:20 |
Farooq |
the only potentional drawback I see in my idea is what Krock said about ABI compatability. It's something I have never worked with outside Linux and x86_64 |
| 09:20 |
repetitivestrain |
as regards the rest, the map generator on my server does not exercise the built in decoration system, l-system trees, the built-in ore generator, any of the built-in ValueNoise facilities, etc. |
| 09:20 |
repetitivestrain |
in a word, it is completely indifferent to the existence of the facilities you propose to remove |
| 09:21 |
Farooq |
exactly! |
| 09:21 |
repetitivestrain |
why then is it necessary to _remove_ existing facilities the better to support lua map generators? |
| 09:21 |
repetitivestrain |
or other programs which do not require them? |
| 09:21 |
repetitivestrain |
i don't see the imperative here |
| 09:23 |
Krock |
also consider that every module would come with a significant overhead of imported symbols (function/variables), either from the C/C++ standard library or the Luanti engine |
| 09:24 |
Farooq |
nope I'm not proposing "removing" them. I'm proposing that ideally, the engine is modular. And these are modules like other modules in the game. The engine can come with a starterpack of these modules if most games require it. But there should be also a possibility for people like you to not have them when they don't need them. |
| 09:24 |
repetitivestrain |
why not |
| 09:24 |
repetitivestrain |
it's plenty of complexity for questionable or little gain |
| 09:24 |
Krock |
and if C++ is used, then most of the functions get baked into the module - again. this includes the std::vector, std::map and whatnot implementations. especially stringstream can bloat it considerably. |
| 09:25 |
Farooq |
Aren't these part of standard C++ library? |
| 09:25 |
Krock |
the C++ standard library is mostly headers-only. |
| 09:25 |
repetitivestrain |
and such facilities as would actually require server or mod implementers to patch the engine couldn't possibly be accommodated by a general purpose and stable module API anyway |
| 09:26 |
Farooq |
I mean isn't that a dynamic library. If my map generator X requires standard C++ library and so does the engine, it is loaded once. Isn't it? |
| 09:26 |
repetitivestrain |
see https://github.com/luanti-org/luanti/pull/16429 |
| 09:26 |
Farooq |
Unless you are using a static library |
| 09:26 |
repetitivestrain |
no, the C++ stl is inlined into its consumers |
| 09:27 |
Farooq |
Note that I haven't got almost any experience with C++ |
| 09:27 |
repetitivestrain |
a copy of a substantial amount of code generally exists in every object file which includes it |
| 09:27 |
Farooq |
That's a problem. But also consider that your custom map generator module might not be written in C++ |
| 09:28 |
Krock |
Farooq: consider decompiling Luanti, preferably with debug symbols. A lot will be generated code from the C++ headers |
| 09:29 |
Krock |
s/A lot/A portion/ |
| 09:30 |
Farooq |
Something here I don't understand. I haven't worked with C/C++ for so long. But as far as I remember, headers are (mostly) not the code to be executed. Am I right? |
| 09:31 |
Krock |
https://stackoverflow.com/a/3482966 |
| 09:32 |
Farooq |
yeah exactly. except I didn't know the difference is cultural |
| 09:35 |
Farooq |
Do you think core developers would welcome a PR or at least a suggestion for experimentally adding, loading and executing dynamic modules for the patches of JMA server |
| 09:35 |
Farooq |
s/server/server?/ |
| 09:35 |
Farooq |
my mind is now fried and at the same time I need food |
| 09:35 |
Farooq |
bbl |
| 09:38 |
MTDiscord |
<josiah_wi> Farooq, in C++, header files also contain a substantial amount of source code sometimes. |
| 09:43 |
MTDiscord |
<josiah_wi> However, for your purposes, I do not think that is necessarily a problem. You might want to remove unnecessary code from the binary, but it looks like that is doable. |
| 09:45 |
MTDiscord |
<josiah_wi> Loading dynamic modules in the engine might be perceived as a risk for general users. Perhaps general users aren't responsible enough to avoid loading up dynamic modules that happen to be malware. |
| 10:29 |
|
SFENCE joined #luanti-dev |
| 10:42 |
MTDiscord |
<herowl> sfan5: specifying humidity/heat map would certainly be useful... even more useful would be ability to specify a general heightmap that would offset the noise |
| 10:48 |
|
Farooq joined #luanti-dev |
| 10:49 |
Farooq |
josiah_wi, I agree about the risk. I was more thinking to servers than clients. Another option might be WebAssembly which is said to have only 10% overhead comparing to native. And also so many languages can be compiled into it. |
| 10:56 |
Farooq |
If I limit scope of what is needed, we are able to judge things better. For instance let's consider any module which needs to do read information from Luanti like map, players, chat and so on, compute and return something. For let's consider core.find_path(). If it can be implemented in Lua without trouble and hassle and with good enough performance, then it shouldn't have been added to the engine. Now for the sake of thought experiment, let's consider |
| 10:56 |
Farooq |
it's not possible. The engine should have some mechanism of loading an external native or WASM code, give it the information it wants and read the information it has computed from it. One way for sure is dynamic libraries. Another way is TCP/UDP sockets. Another way is standard stdout and PIPE. |
| 10:57 |
Farooq |
For now I want to add this thing to my TODO list. In future, I would check why JMA server requires patches to the engine. And how a modular mechanism could be implemented. |
| 11:00 |
Farooq |
I want to get my hands dirt with engine code for a very long time. And now I have the excuse to do it. |
| 11:01 |
|
Farooq joined #luanti-dev |
| 11:08 |
|
SFENCE_ joined #luanti-dev |
| 11:10 |
sfan5 |
03:09 <[MatrxMT]> <repetitivestrain> It would be useful if custom mapgens could implement get_biome_data and provide biome maps from Lua |
| 11:11 |
sfan5 |
it sure would but pretty much anyone at any time can call core.get |
| 11:11 |
sfan5 |
_heat or get_humidity and funelling those calls into callback in a single thread(?) is annoying and also a bottleneck |
| 11:11 |
sfan5 |
hence the suggestion to specify everything ahead of time |
| 11:11 |
repetitivestrain |
well my biome system has proved efficient enough to support all uses of get_biome_data that exist in mineclonia |
| 11:12 |
repetitivestrain |
and it does run independently in mapgen, async, and server environments |
| 11:12 |
repetitivestrain |
just my two cents as someone who has recently been implementing custom biome systems |
| 11:13 |
sfan5 |
which resolution does your biome system need to work nicely? as in "I can decide the biome every N nodes" |
| 11:13 |
sfan5 |
<Farooq> Do you think core developers would welcome a PR or at least a suggestion for experimentally adding, loading and executing dynamic modules for the patches of JMA server |
| 11:14 |
repetitivestrain |
sfan5: biome positions are quantized to multiples of 4 |
| 11:14 |
repetitivestrain |
and it's the same in minecraft for what it's worth |
| 11:14 |
sfan5 |
without knowing what exactly JMA needs this sounds prone to ending up with a bunch of hooks that are useful to them but nobody else |
| 11:15 |
sfan5 |
repetitivestrain: sure but what is the least resolution that is acceptable? |
| 11:15 |
Farooq |
sfan5, I should really write a detailed proposal |
| 11:15 |
Farooq |
It seems people in chat don't understand my point |
| 11:16 |
repetitivestrain |
sfan5: i'd have to say 4x4, perhaps 6x6, since biomes must follow the contours of rivers and other water bodies, and 4x4 already cuts it rather fine in that regard |
| 11:18 |
|
turtleman joined #luanti-dev |
| 11:18 |
sfan5 |
6x6 resolution would require 230 MB to specify it ahead of time for the entire world |
| 11:19 |
sfan5 |
which is not clear if it's practical for you anyway |
| 11:19 |
repetitivestrain |
i suspect it would take much longer to generate that much data |
| 11:19 |
repetitivestrain |
i can actually attempt to measure this, one moment |
| 11:20 |
repetitivestrain |
(and note that biomes are partially 3d too, which get_biome_data can account for but the current biomemap system cannot) |
| 11:22 |
Krock |
is it necessary to define ahead of time? |
| 11:23 |
Krock |
I mean the entire map. chunk-wise could work too. |
| 11:25 |
sfan5 |
core.get_heat() can be called at any time and that was my idea to fix this in an uncomplicated way |
| 11:26 |
sfan5 |
but I guess that doesn't work |
| 11:48 |
|
SpaceManiac joined #luanti-dev |
| 13:06 |
luatic |
!tell Farooq (1) core.find_path has existed for what, a decade? and barely been touched. it's not a new addition. (2) new additions *are* adding lower level, general purpose APIs. we *are* dehardcoding. (3) esp. minor forks are fine. (4) if modsec is off, you can load dynamic libraries from lua. |
| 13:07 |
sfan5 |
use ~tell here |
| 13:07 |
luatic |
~tell Farooq (1) core.find_path has existed for what, a decade? and barely been touched. it's not a new addition. (2) new additions *are* adding lower level, general purpose APIs. we *are* dehardcoding. (3) esp. minor forks are fine. (4) if modsec is off, you can load dynamic libraries from lua. |
| 13:07 |
ShadowBot |
luatic: OK. |
| 14:10 |
repetitivestrain |
sfan5: generating biomes for all of one map took 1 hr and 14 minutes, but only the 2d version |
| 14:10 |
repetitivestrain |
on a single thread, which is all that's feasible in lua |
| 14:11 |
repetitivestrain |
however, since the depth value varies by the base terrain offset noise, it would be necessary to sample this noise vertically at increments of 4/6 nodes too for each column, to handle underground biomes and suchlike |
| 14:32 |
Farooq |
luatic, thanks for letting me know. I never saw it before |
| 14:55 |
MTDiscord |
<josiah_wi> For the record I don't personally hold the opinion that users should be prohibited from downloading and running C++ modules for their own protection. |
| 15:06 |
luatic |
Ultimately they are not - if they trust a mod, or disable mod security, they can load shared libraries from Lua, for example - but sandboxing is a legitimate feature of Lua. |
| 15:06 |
repetitivestrain |
i think szutilpack is on contentdb and also provides native modules |
| 15:07 |
repetitivestrain |
although users are required to compile them themselves |
| 15:07 |
repetitivestrain |
also core.find_path hasn't been touched because it's never worked |
| 15:07 |
repetitivestrain |
do your users a favor and avoid it like the plague |
| 15:23 |
|
rubenwardy joined #luanti-dev |
| 16:40 |
|
cheapie joined #luanti-dev |
| 16:57 |
|
SFENCE joined #luanti-dev |
| 17:12 |
|
fluxionary joined #luanti-dev |
| 17:17 |
|
Desour joined #luanti-dev |
| 17:56 |
|
v-rob joined #luanti-dev |
| 18:48 |
|
SFENCE joined #luanti-dev |
| 19:07 |
|
SFENCE joined #luanti-dev |
| 21:05 |
|
ivanbu joined #luanti-dev |
| 22:09 |
|
YuGiOhJCJ joined #luanti-dev |
| 22:33 |
|
panwolfram joined #luanti-dev |
| 23:05 |
|
Eragon joined #luanti-dev |
| 23:38 |
|
YuGiOhJCJ joined #luanti-dev |
