Time Nick Message 01:08 ireallyhateirc2 I'l trying to build the latest master branch and get this in tests: 01:08 ireallyhateirc2 (server) 2025-04-13 01:04:16: ERROR[Server]: ...40.drv-0/source/games/devtest/mods/unittests/version.lua:11: assertion failed! 01:08 ireallyhateirc2 (server) [FAIL] test_get_version - 0ms 01:09 ireallyhateirc any clues why this happens? 09:14 [MatrxMT] seems like the version hash is not a string 09:14 [MatrxMT] why didn't you look at the code? 09:15 [MatrxMT] nevermind 09:20 [MatrxMT] please ignore what i said 12:05 ireallyhateirc I tried the new spatial cache thing with Exile. I didn't look at any numbers, mostly only checked if I can visually feel the difference. 12:06 ireallyhateirc And the answer is mostly no. Entities usually lag on mapblock updates and our game has a lot of them. 12:11 ireallyhateirc surprisingly the legboat on mtg appears to be smoother than last time I checked but I also bought an SSD since then 12:12 ireallyhateirc but there's still that noticeable and periodic lag where the legs of the legboat lag behind, I believe it happens when the server generates a mapchunk and then emerges blocks 12:27 ireallyhateirc The thing is probably either https://github.com/luanti-org/luanti/issues/15125 or https://github.com/luanti-org/luanti/issues/14018 12:29 ireallyhateirc Or is it because there's no time synchronization between client and server? In Exile for example entities get teleported back and forth when the engine appears to be busy doing something (most likely block emerging) 12:30 sfan5 block emerging does not happen on the main thread so you need to ask yourself what is blocking it 12:33 ireallyhateirc are ABMs working on the main thread? 12:34 sfan5 yes 12:36 ireallyhateirc well we're processing huge VMs on the main thread and our entities constantly read nodes around them. As for the reduced but still noticeable lags with legboat + mtg then idk what could be blocking there 12:37 sfan5 the engine can't fix lag sources of your own making 12:38 ireallyhateirc sfan5, yeah I'm aware. I did not know block emerging should not affect the main thread. 12:39 sfan5 what the spatial cache PR is about is preventing (additional) server lag when you have lots of entities. it will do literally nothing for a simple "jump into a boat in singleplayer and see if it's smooth" case 12:40 ireallyhateirc maybe it's just the HDD -> SSD upgrade that made it smooth 12:40 ireallyhateirc or something else 12:42 ireallyhateirc My reasoning was that since such a simple mod (legboat + mtg) produces similar lags to Exile then the underlying cause would be engine-related 12:46 ireallyhateirc I do need to fix the crappy code I wrote though, that's a sure thing. 12:55 ireallyhateirc sfan5, I started the very same legboat world on the HDD and the boat lags more 12:56 sfan5 i didn't say that map loading would not affect the main thread, just that it (mostly) doesn't run in it 13:00 ireallyhateirc so it's mostly "Busy Disks Cause Lag #15125" then? I don't recall playing other games that would lag so much on a HDD 13:00 ShadowBot https://github.com/luanti-org/luanti/issues/15125 -- Busy Disks Cause Lag 13:00 ireallyhateirc I doubt the inverse kinematics calculations for the boat are so heavy that we can call it bad code 13:01 MTDiscord Try to exhaust your disk IOPS and see if it lags more? 13:01 MTDiscord we need some numbers 13:02 sfan5 there's far from enough information to even make a guess 13:03 ireallyhateirc how do I get the info you need? 13:04 MTDiscord some basic things you can do: what is max_lag? could also use the lagometer (https://gitlab.com/sztest/szutilpack/-/tree/master/szutil_lagometer) 13:05 MTDiscord then, you can profile. limitation of that is that you just see the aggregate of everything, buf if something is consistently bogging it down, it should be visible in the flamegraph. 13:06 MTDiscord for profiling the engine, see https://github.com/luanti-org/luanti/blob/master/doc/developing/misc.md#profiling-luanti-on-linux-with-perf. you should ideally profile client and server separately. i think in your case only the server needs profiling. 13:07 MTDiscord for profiling lua, see https://content.minetest.net/packages/jwmhjwmh/jitprofiler/. 13:07 ireallyhateirc as for the charts I get when pressing F5 - for the FPS chart the upper number is max FPS and the lower is min FPS ? 13:07 ireallyhateirc and the dips mean lagspikes? 13:08 MTDiscord well is this a client problem or a server problem? 13:08 MTDiscord is your FPS dipping or not? FPS should be essentially independent of the server. 13:08 MTDiscord independent of server lag, that is 13:08 ireallyhateirc The client has usually 60 FPS then the boat lags and it causes a dip to 30-40 just for a fraction of a second 13:12 ireallyhateirc and the dips usually happen when there's a spike in packets received by the client 13:13 MTDiscord so it is a client lag problem 13:14 ireallyhateirc I'm installing the lagometer 13:14 MTDiscord note that the lagometer is for server lag 13:16 ireallyhateirc if there's no server lag but the boat still lags then it should confirm it's a client-side problem 13:29 ireallyhateirc idk if I'm reading this properly but: SSD - max lag 0.216, HDD - max lag 0.613 13:30 ireallyhateirc (from the lagometer) 13:30 ireallyhateirc I'd like to note that Luanti is installed on the SSD and I only move the world to the HDD 13:44 MTDiscord what are the non-max-lag statistics (that's kind of the point of the lagometer, the engine already reports max lag) 13:45 ireallyhateirc I can send a screenshot 13:46 ireallyhateirc SSD: https://litter.catbox.moe/qp62du.png 13:46 ireallyhateirc HDD: https://litter.catbox.moe/ulgy9z.png 13:47 ireallyhateirc idk how to read all that 13:49 ireallyhateirc Oh it's explained in the documentation 13:51 ireallyhateirc mean time between steps appears to be higher on HDD: 0.029 (SSD) 0.059 (HDD) 14:00 ireallyhateirc I love spending sundays on walking around in my legboat 14:14 Krock ireallyhateirc: your boat is kinda cursed 14:14 ireallyhateirc Krock, that's the point of the legboat 14:14 [MatrxMT] that boat got bitten by a radioactive spider 15:07 MinetestBot 02[git] 04rubenwardy -> 03luanti-org/luanti: ContentDB: Add reviews tab (#15254) 1375862e3 https://github.com/luanti-org/luanti/commit/75862e33b6bfe6b3805310152c66af1430ac3666 (152025-04-13T15:07:01Z) 15:16 ireallyhateirc I'll come back with the profiler's output later. My computer is not a potato and I've played more fancy games on it without such lags with HDDs so I believe there's something indeed lurking in the engine. 15:26 ireallyhateirc Should I start an issue for the unconfirmed bug? 16:07 ireallyhateirc I know it's like comparing apples to oranges but in Supertuxkart I don't get any lags like that. If there's a Luanti game with moving entities that don't lag then send me a link and I'll test it 16:12 sfan5 the sleep times in the graphs are suspicious, or did you put the game into the background before making the screenshot? 16:13 ireallyhateirc sfan5, I used the system's screenshot thing instead of luanti's one 16:13 ireallyhateirc I can redo it with luanti's screenshot thing 16:13 sfan5 is that a yes or no 16:14 ireallyhateirc I'm not sure. When I press the "print screen" a window pops up and defocuses the window 16:14 ireallyhateirc so probably yes? 16:15 ireallyhateirc Luanti switches to idle mode when that happens so yes, I'll redo the test 16:17 sfan5 then that's unrelated 16:17 sfan5 either way I can hop into a MTG boat and drive to newly generated parts without a single hitch 16:19 ireallyhateirc I'll test the boat without legs first then 16:29 ireallyhateirc Ok so with a regular boat on a HDD the lags are less noticeable, but I still do get them. Max lag is at ~0.500 and FPS are dropping from time to time to 30-50 16:30 sfan5 aside from "lag" being too vague you're mixing things up again: max_lag is server-side, fps is client-side 16:33 ireallyhateirc I know, I'm just giving the numbers I can see. On the SSD: max lag ~0.100, FPS sometimes drop to 50 16:38 ireallyhateirc so whatever it is, it appears to be affected by the storage device's parameters the world files are on 16:38 ireallyhateirc and I'm not mining crypto or anything, Luanti is the sole user of the HDD at a given moment 16:39 ireallyhateirc I can provide better numbers and a video of both cases, should I start an issue for this? 16:44 sfan5 a video is good in any case so people can precisely tell what the lag looks like 16:57 ireallyhateirc Here's one: https://files.catbox.moe/cpiap4.mp4 16:57 ireallyhateirc the first lag is around 00:18 16:58 ireallyhateirc it looks as if the boat was moved back and forth 17:00 ireallyhateirc Then another saucy one is around 00:43 17:02 cheapie Max lag of 0.4 seems kinda high if you're expecting entities to be smooth 17:03 cheapie Does the server profiler point at some specific mod as being a problem? 17:08 whosit if you're using sqlite backends, try setting sqlite_synchronous = 0 17:08 ireallyhateirc cheapie, that's vanilla MTG 17:09 cheapie Being vanilla MTG doesn't exclude the possibility of a bug in MTG 17:10 ireallyhateirc whosit, where do I put that setting? 17:11 whosit in minetest.conf 17:11 whosit or, luanti.conf, idk which one you're using :p 17:12 sfan5 the boat jumping back to a past position for exactly one frame is unlike anything a server lag should cause 17:13 whosit can it be caused by packets arriving out of order somehow? or those are reliable and ordered? 17:13 whosit (or being processed out of order? :p) 17:14 sfan5 it can 17:17 ireallyhateirc whosit, appears to be way smoother 17:18 whosit if it does, you can also try this: https://phiresky.github.io/blog/2020/sqlite-performance-tuning/ 17:18 ireallyhateirc which is weird because max lag is the same and the client's FPS chart also shows FPS dips to 50 or 40 17:19 whosit saw someone link it, somewhere, but I think main issue is the disk sync 17:19 ireallyhateirc or not? This appears to happen when new chunks are generated 17:20 ireallyhateirc nah, same story - new chunks new lags 17:20 ireallyhateirc Hope that the video at least shows that I did not make this up lol 17:21 sfan5 is this singleplayer or networked? 17:22 ireallyhateirc singleplayer 17:28 ireallyhateirc I'll be back later and I'll create an issue for this then 17:29 ireallyhateirc the HDD is "2TB WDC WD20EZRZ-22Z" and SSD is "1024GB SSDPR-CX400-01T-" btw if you're curious about the specs 17:35 cheapie Searching for "SSDPR-CX400-01T-" just gets me those SSDs people buy off Aliexpress for like $20, the ones that usually work but manage to be slower than a hard drive 18:21 ireallyhateirc it wasn't that cheap lol 18:21 ireallyhateirc https://www.goodram.com/produkty/ssd-cx400-sata-iii-25/ 18:21 ireallyhateirc It was 50$ or so? idk 18:49 ireallyhateirc chasing elusive bugs is my passion 20:50 ireallyhateirc Could be better: https://github.com/luanti-org/luanti/issues/16019 20:50 ireallyhateirc I'll provide profiler output later once I know how to do that 21:02 tsadok I have been avoiding SSDs due to some really bad experiences early on, that are *probably* out of date at this point because new tech gets better over time, but eh, spinning-platter drives have become *so* much more reliable over the last three decades, that I no longer have really any interest in trusting my data to a different storage technology. 21:04 ireallyhateirc that particular brand/model I use is decent and hasn't caused me any problems so far 21:05 ireallyhateirc I have some ugly experience with NVMe SSDs. One became read-only after my computer experienced unexpected shutdown 21:12 cheapie Random nonsense with replacing nodes with others that shouldn't be there: https://cheapiesystems.com/media/2025-04-13%2016-07-32.webm 21:14 ireallyhateirc the ultimate elevator 21:22 MTDiscord the ultimate elevator is a 2 wide shaft that goes straight down, with one column for water, the other for air, and some air pockets every 20 nodes or so. then you can use a boat to go up fast, and drop down and use the water to slow down even faster. and it's very cheap to build. 21:23 cheapie luatic: Yeah, but can that do this? https://cheapiesystems.com/media/2025-03-21%2003-23-07.webm 21:24 MTDiscord i must concede 21:25 cheapie Or have keyswitches that just confuse everyone since hardly anyone knows what they do IRL either? :P https://cheapiesystems.com/media/images/screenshot_20250404_225700.png 21:32 ireallyhateirc can someone send me an example of a sane flamegraph from the jit profiler? 21:32 ireallyhateirc I get weird and broken results