| Time |
Nick |
Message |
| 00:54 |
|
ireallyhateirc joined #luanti |
| 01:38 |
|
FileX joined #luanti |
| 01:52 |
bgstack15 |
Multiplexing serverlist is even easier than contentdb. I've got a working model, but need to clean up code before adding it to the public copy of contentdbproxy. |
| 01:58 |
bgstack15 |
I was thinking of renaming my project LuantiWebServicesProxy, lwsp. Because it would be a multiplexer of both ContentDB and Serverlist. A client can use either or both parts. |
| 02:21 |
|
Can0xfBows joined #luanti |
| 02:51 |
|
aliasstilltaken joined #luanti |
| 04:00 |
|
MTDiscord joined #luanti |
| 04:43 |
|
Haven0320-2 joined #luanti |
| 05:01 |
|
FeXoR joined #luanti |
| 05:13 |
|
SwissalpS joined #luanti |
| 05:41 |
|
identity joined #luanti |
| 05:54 |
|
sparky4 joined #luanti |
| 06:43 |
|
kamdard joined #luanti |
| 06:43 |
|
SwissalpS joined #luanti |
| 06:51 |
|
Trifton joined #luanti |
| 07:01 |
|
Trifton_ joined #luanti |
| 07:11 |
|
cow321 joined #luanti |
| 07:18 |
|
Trifton joined #luanti |
| 09:31 |
|
mrkubax10 joined #luanti |
| 10:36 |
|
Desour joined #luanti |
| 10:45 |
|
Sompi joined #luanti |
| 11:05 |
|
pgimeno_ joined #luanti |
| 11:34 |
|
stg-developer joined #luanti |
| 12:15 |
|
PoochInquisitor joined #luanti |
| 13:27 |
|
jaca122 joined #luanti |
| 14:02 |
|
CRISPR joined #luanti |
| 14:38 |
bgstack15 |
I got the LuantiWebServicesProxy repo up: https://codeberg.org/bgstack15/lwsp |
| 14:40 |
pgimeno |
<repetitivestrain> LuaJIT's garbage collector is completely unfit for a real-time video game |
| 14:40 |
pgimeno |
Not true. First, it's the same as PUC Lua, so it makes no sense to single out LuaJIT here. Second, take a look at Gravity Circuit, an action platformer game made with Love2D. It certainly takes awareness of the issue in order to avoid unnecessary allocations (e.g. not creating functions on the fly, not creating ad-hoc single-use tables for parameters, etc.) and sometimes, manually running the GC or fine-tuning the GC parameters to be more aggressive. |
| 14:43 |
repetitivestrain |
pgimeno: i don't know much about PUC Lua, but it is absolutely true. take any minetest game which is free of bottlenecks elsewhere and you will observe that long globalsteps are either dominated by garbage collection or by ABM execution. a good garbage collector such as exist in the JVM can sustain multiple gigabytes of allocations per second in tiny objects while incurring sufficiently little overhead to allow rendering 144 frames |
| 14:43 |
repetitivestrain |
per second |
| 14:45 |
pgimeno |
I'm horrified every time I look into any mod's code, because the mod programmers are totally unaware of this. That doesn't make LuaJIT unfit for a real-time video game. |
| 14:45 |
sfan5 |
"you have to work around several core language features that will break your neck" is not what I'd call "fit for a real-time video game" |
| 14:47 |
sfan5 |
but it's true that it's not impossible |
| 14:49 |
pgimeno |
Lua is beginner friendly, but you can't expect a beginner to be able to pull a real-time video game. It takes a deeper knowledge of the language's limitations to become able to do that, is my opinion. |
| 14:49 |
MTDiscord |
<luatic> pgimeno: I'm not sure whether it is fair to compare 2d with 3d games. the latter usually require an order of magnitude (or more) more computational resources since many things scale cubically instead of quadratically. |
| 14:49 |
repetitivestrain |
https://ibb.co/gbGX2HTG |
| 14:51 |
repetitivestrain |
the language runtime on which a certain voxel game which shall remain unnamed is implemented, for instance, can sustain 2 GB/s of allocations and while constantly performing garbage collection without ever falling behind my display's refresh rate |
| 14:51 |
repetitivestrain |
i would give almost anything for this degree of flexibility in my code |
| 14:51 |
repetitivestrain |
(my lua minetest code, that is) |
| 14:52 |
repetitivestrain |
because, when last i benchmarked a server with thousands of mobs and numerous additional objects, most of the garbage collection overhead was incurred by the engine allocating vectors and moveresult tables for on_step callbacks |
| 14:53 |
MTDiscord |
<luatic> Some Java GCs are crazy good, that is true |
| 14:53 |
pgimeno |
@luatic there are 3D games made with love2d, you know... |
| 14:54 |
sfan5 |
i think that shows more how much work went into the jvm gc and not that java is a good language for a real time game ;) |
| 14:54 |
sfan5 |
repetitivestrain: a big problem for engine reuse is that mods can't promise not to modify a given argument |
| 14:54 |
MTDiscord |
<luatic> pgimeno: I'm aware, but how many are actually fast? Is there a single fast voxel game in love 2d? |
| 14:54 |
sfan5 |
or well they can but there's no language feature to enforce this |
| 14:55 |
repetitivestrain |
yes, exactly, and this is one of the many reasons i don't think lua is optimal for real-time video games |
| 14:55 |
[MatrxMT] |
<Blockhead256> just push an error state back to Lua if they do >:^) |
| 14:56 |
sfan5 |
lua is great for light business logic |
| 14:56 |
pgimeno |
@luatic and Luanti mods are not a 3D game, the 3D part is the engine's task, leaving the scripting parts to Lua(JIT). And about your question: https://github.com/groverburger/lovecraft (again that's Lua fully dedicated to doing the 3D, which is quite a feat) |
| 14:56 |
repetitivestrain |
but how performant is it? |
| 14:57 |
sfan5 |
in any case we're kinda stuck with lua right now so if you want it fixed there better be an idea on how |
| 14:57 |
MTDiscord |
<luatic> Don't get me wrong, love2d is a lovely framework. But I do think (and have observed myself) that you'll run into performance limitations when you start doing computationally heavy tasks like mapgen. |
| 14:57 |
pgimeno |
the program in that link is quite fast, 60fps in my slow card IIRC |
| 14:57 |
sfan5 |
maybe ffi can fix this, somehow |
| 14:58 |
repetitivestrain |
yeah if you read the context i mentioned these bottlenecks by way of encouragement for mod authors to be conscious of memory allocation |
| 14:58 |
MTDiscord |
<luatic> We should explore FFI for VoxelManip |
| 14:59 |
MTDiscord |
<luatic> A lazy API could just return nil for non LuaJIT, placing the burden of a reasonable fallback for PUC on the modder |
| 14:59 |
sfan5 |
what would personally interest me is an answer to: what is the biggest current Lua/API bottleneck that is caused by the engine and cannot be worked around? |
| 14:59 |
pgimeno |
ffi does go a long way with , see https://notabug.org/pgimeno/ffi_accel and #6863 which sadly was rejected |
| 15:00 |
ShadowBot |
https://github.com/luanti-org/luanti/issues/6863 -- Add LuaJIT FFI-friendly memory-intensive functions |
| 15:00 |
repetitivestrain |
avoiding allocations of temporary objects is not simple (and i speak from experience, having written a map generator which allocates nearly all arrays and data ahead of time, and allocates perhaps two or three dozen tables/mapblock for generation notifications, which is one factor behind its performing so well for a lua mapgen of its complexity) |
| 15:00 |
pgimeno |
*with mapgen |
| 15:00 |
repetitivestrain |
sfan5: the greatest mapgen bottleneck is not a bottleneck but a limitation that produces excessive memory consumption, and it is the absence of any means of accessing a voxelmanip's mapnode array directly from lua |
| 15:01 |
repetitivestrain |
the greatest gameplay bottleneck is the sheer number of tables that are allocated for object on_step callbacks and the fact that numerous apis, such as object:get_pos, return vectors |
| 15:02 |
sfan5 |
which of those two would you want fixed first? |
| 15:02 |
repetitivestrain |
the mapgen issue |
| 15:03 |
pgimeno |
#6863 + FFI would have totally mooted the first one |
| 15:03 |
ShadowBot |
https://github.com/luanti-org/luanti/issues/6863 -- Add LuaJIT FFI-friendly memory-intensive functions |
| 15:03 |
repetitivestrain |
i have mostly resigned myself to the other issue, though an object:get_pos_raw function which returns object positions on the stack rather than as a vector would be very helpful |
| 15:04 |
pgimeno |
btw the code is still here: https://notabug.org/pgimeno/minetest/pulls/2/files |
| 15:04 |
sfan5 |
could the engine simply return an ffi array and safely allow mods to work with it? |
| 15:04 |
sfan5 |
optimizations that only work without mod security are basically useless |
| 15:04 |
repetitivestrain |
sfan5: unfortunately the ffi is not secure at all |
| 15:07 |
pgimeno |
sfan5: it already does, sort of, the problem is that the interface is C++ instead of C, which isn't accessible by FFI in practice, hence the PR |
| 15:08 |
pgimeno |
yeah, an FFI array is not secure, it's a raw pointer |
| 15:09 |
pgimeno |
even the ffi_accel mod, which tried to limit access through functions, is unsafe |
| 15:09 |
repetitivestrain |
you could wrap it with a bounds check, but i think the debug.* functions could be abused to manipulate code in the process of performing such tests |
| 15:10 |
sfan5 |
ah yeah the docs say so: >The FFI library provides no memory safety, unlike regular Lua code. It will happily allow you to dereference a NULL pointer, to access arrays out of bounds or to misdeclare C functions. If you make a mistake, your application might crash, just like equivalent C code would |
| 15:10 |
pgimeno |
debug.* requires an insecure environment, right? |
| 15:10 |
repetitivestrain |
i don't know |
| 15:11 |
pgimeno |
anyway, I don't remember much but I do remember that I attempted to make ffi_accel as secure as was feasible |
| 15:11 |
pgimeno |
but there were still security holes remaining |
| 15:13 |
repetitivestrain |
ultimately with lua you have to cut your losses and either compromise on safety or obtain a better computer, and even then without really attaining performance comparable to the JVM (which is luanti's foremost competition i think) or native code with static memory management |
| 15:14 |
|
qqe joined #luanti |
| 15:14 |
sfan5 |
anyway that means ffi is off the table |
| 15:14 |
repetitivestrain |
and that is why it is not optimal for real-time video games, though it is not that luanti has much of a choice in the matter |
| 15:15 |
sfan5 |
next best thing I can think of is integrating wasm and then letting wasm code interact with API using native data types |
| 15:16 |
repetitivestrain |
or maybe i'm making a mountain out of a molehill in insisting on <=50 ms globalsteps |
| 15:17 |
repetitivestrain |
since the linux forks people have 1 second globalsteps as a matter of routine |
| 15:17 |
repetitivestrain |
and they don't appear to have suffered much for it |
| 15:17 |
[MatrxMT] |
<Blockhead256> lol it's true |
| 15:18 |
repetitivestrain |
but mineclonia is generally more sensitive to server lag, as we have the lowest dtime target of any luanti game i've seen thus far |
| 15:18 |
[MatrxMT] |
<Blockhead256> orwell had to work out how to make advtrains move reliably under server load. Railwaytime means that even if the server globalsteps are ramping up in length, the trains get a fair share of processing and don't get out of sync |
| 15:18 |
repetitivestrain |
50 ms |
| 15:18 |
repetitivestrain |
and with dtimes above 75 ms mob physics begin to malfunction |
| 15:19 |
repetitivestrain |
Blockhead256: how are 1 second dtimes tolerable though |
| 15:20 |
[MatrxMT] |
<Blockhead256> well you see, for starters, my RTT to lifo is 330 ms |
| 15:20 |
repetitivestrain |
i imagine it should produce plenty of jank when attached to entities and so forth |
| 15:26 |
pgimeno |
I don't get why FFI is off the table and built-in mapgens not. FFI code can go out of bounds; so can engine code. If you trust the engine, you can trust a mapgen, right? |
| 15:26 |
sfan5 |
no because the Lua mod code is untrusted |
| 15:27 |
repetitivestrain |
well insecure-only optimizations would be acceptable for me too |
| 15:28 |
repetitivestrain |
considering that i've already implemented several optional ones, more couldn't hurt |
| 15:28 |
pgimeno |
well, it is untrusted unless the user trusts it... Android lets you run untrusted code too, you just have to accept permission. |
| 15:29 |
repetitivestrain |
e.g. debug.upvaluejoin in combination with dumping and loading functions enables you to copy a luajit GCproto bytecode object and assign each closure a unique GCproto |
| 15:29 |
[MatrxMT] |
<Blockhead256> the myth of consensual trust: User: I trust my OS. OS: I trust my user. Microsoft: Isn't there someone you forgot to ask? |
| 15:29 |
[MatrxMT] |
<Blockhead256> also, isn't Android moving to block unsigned APKs? |
| 15:29 |
repetitivestrain |
which prompts luajit consistently to specialize recorded instructions to the values of upvalues in the closure |
| 15:30 |
pgimeno |
lol, power users are an endangered species |
| 15:30 |
repetitivestrain |
and yields a healthy 10% speed improvement, but requires that mod security be disabled so that functions may be dumped and loaded from strings |
| 15:30 |
sfan5 |
it can be put behind secure.trusted_mods for sure but then it's not useful to most users |
| 15:31 |
sfan5 |
and we don't want to train users to grant full access to their computer to random mods "because it's too slow otherwise" either |
| 15:31 |
repetitivestrain |
i'm a bit skeptical of that, since a map generator is unquestionably a very significant mod which users will display interest in optimizing |
| 15:32 |
repetitivestrain |
particularly on large servers or slower computers |
| 15:32 |
[MatrxMT] |
<Blockhead256> adding mods to the trusted list is one thing, but disable security entirely leaves them open to the next thing which they may not even realise needs security off to become malware |
| 15:33 |
repetitivestrain |
unfortunately mapgen scripts cannot exploit the trusted list |
| 15:33 |
repetitivestrain |
for core.request_insecure_environment just doesn't function in mapgen environments |
| 15:33 |
repetitivestrain |
or doesn't exist, but i don't remember which |
| 15:34 |
sfan5 |
ah yeah there was that |
| 15:34 |
pgimeno |
maybe secure.trusted_mods could be maintained by a menu option to grant permission to a mod that requires it... e.g. declare in mod.conf if it is required, then when a mod requires insecure environment, ask the user whether to grant permission |
| 15:35 |
pgimeno |
of course the dialogue to accept permission should be damn scary and encourage non-acceptance |
| 15:35 |
repetitivestrain |
i am also _optionally_ addressing a number of luajit bottlenecks with the FFI |
| 15:35 |
sfan5 |
pgimeno: yeah the risk of "what is this, let me click 'OK' to get rid of it" is very real |
| 15:36 |
[MatrxMT] |
<birdlover32767> what about double-confirmation? |
| 15:36 |
sfan5 |
by not having a consent dialog we're (for better or worse) we're avoiding that risk |
| 15:36 |
[MatrxMT] |
<birdlover32767> or is a scary red dialogue ok |
| 15:36 |
repetitivestrain |
if a server operator (such as yours truly) desires, it can be enabled by compiling a C file as a shared object, disabling mod security, dropping ffi.so into the mod folder, and toggling a setting |
| 15:36 |
repetitivestrain |
https://codeberg.org/mineclonia/mineclonia/src/branch/mineclonia_mapgen/mods/MAPGEN/mcl_levelgen/ffi.c |
| 15:37 |
sfan5 |
what are those __GNUC__ checks for? looks fishy |
| 15:37 |
repetitivestrain |
GCC guarantees arithmetic rshifts |
| 15:37 |
repetitivestrain |
while certain other C compilers don't |
| 15:38 |
|
peterz joined #luanti |
| 15:39 |
repetitivestrain |
the ffi itself is also used to eliminate array bounds checks and indirection while accessing the gradient list and indexing permutation arrays: https://codeberg.org/mineclonia/mineclonia/src/branch/mineclonia_mapgen/mods/MAPGEN/mcl_levelgen/noise.lua#L34 |
| 15:39 |
repetitivestrain |
which, when combined with the C module, nearly doubles performance |
| 15:41 |
repetitivestrain |
also, gcc guarantees that lshifts on signed values operate on their binary representation |
| 15:41 |
repetitivestrain |
and that overflows are well-defined as such |
| 15:42 |
repetitivestrain |
"Bitwise operators act on the representation of the value including |
| 15:42 |
repetitivestrain |
both the sign and value bits, where the sign bit is considered |
| 15:42 |
repetitivestrain |
immediately above the highest-value value bit. Signed ‘>>’ acts on |
| 15:42 |
repetitivestrain |
negative numbers by sign extension. |
| 15:42 |
repetitivestrain |
As an extension to the C language, GCC does not use the latitude |
| 15:42 |
repetitivestrain |
given in C99 and C11 only to treat certain aspects of signed ‘<<’ |
| 15:42 |
repetitivestrain |
as undefined." |
| 15:42 |
|
repetitivestrain was kicked by ShadowBot: Message flood detected. Use a pastebin like pastebin.mozilla.org. |
| 15:42 |
|
repetitivestrain joined #luanti |
| 15:42 |
identity |
should have removed the newlines |
| 15:42 |
repetitivestrain |
ugh, it didn't wrap properly |
| 15:42 |
sfan5 |
I'm sure there are more portable and obvious ways to go about that |
| 15:42 |
repetitivestrain |
i tried |
| 15:43 |
repetitivestrain |
the portable approach is in the !__GNUC__ conditional |
| 15:44 |
sfan5 |
casting to uint? |
| 15:44 |
repetitivestrain |
not if i require sign extension |
| 15:46 |
repetitivestrain |
*lx = (int) (hash << 22) >> 22; *ly = (int) (hash << 12) >> 22; *lz = (int) (hash << 2) >> 22; is equivalent to: arshift (lshift (pos, 22), 22), arshift (lshift (pos, 12), 22),arshift (lshift (pos, 2), 22) |
| 15:46 |
repetitivestrain |
where pos is an aquifer position hash encoding a 10-bit signed offset from a base position |
| 16:45 |
|
Talkless joined #luanti |
| 16:58 |
|
SFENCE joined #luanti |
| 17:05 |
|
mrkubax10 joined #luanti |
| 17:18 |
|
SFENCE joined #luanti |
| 17:24 |
|
sys4 joined #luanti |
| 17:27 |
|
SFENCE joined #luanti |
| 17:31 |
|
pgimeno_ joined #luanti |
| 17:39 |
|
SFENCE joined #luanti |
| 18:13 |
|
SFENCE joined #luanti |
| 18:24 |
MinetestBot |
[git] SmallJoker -> luanti-org/luanti: CGUITTFont: Clean up, unify and comment draw code (#16380) b6a23b1 https://github.com/luanti-org/luanti/commit/b6a23b1bcccccafa7f124310398c0094cd313d0e (2025-09-10T18:23:45Z) |
| 18:24 |
MinetestBot |
[git] sfan5 -> luanti-org/luanti: Add exclude_player to particle spawners f714ac0 https://github.com/luanti-org/luanti/commit/f714ac06116b1725c656aa226688432811cb6f04 (2025-09-10T18:23:55Z) |
| 18:24 |
MinetestBot |
[git] sfan5 -> luanti-org/luanti: Send node dig particles to all other players fcd9c73 https://github.com/luanti-org/luanti/commit/fcd9c73f2f63f93e8d7a0f07ae605fa3d0d467b3 (2025-09-10T18:23:55Z) |
| 18:36 |
|
ireallyhateirc joined #luanti |
| 18:42 |
|
SFENCE joined #luanti |
| 19:01 |
|
SFENCE joined #luanti |
| 19:42 |
|
mrkubax10 joined #luanti |
| 20:19 |
|
silverwolf73828 joined #luanti |
| 20:20 |
|
Haven0320 joined #luanti |
| 20:52 |
|
tzenfore joined #luanti |
| 21:00 |
|
SFENCE joined #luanti |
| 22:05 |
|
Kimapr joined #luanti |
| 22:33 |
|
panwolfram joined #luanti |
| 22:56 |
|
SFENCE joined #luanti |
| 23:05 |
|
Eragon joined #luanti |
| 23:31 |
|
SFENCE joined #luanti |
