| Time |
Nick |
Message |
| 00:16 |
|
TheCoffeMaker joined #minetest-dev |
| 03:01 |
|
SFENCE_arch joined #minetest-dev |
| 05:00 |
|
MTDiscord joined #minetest-dev |
| 07:39 |
|
SFENCE_arch joined #minetest-dev |
| 10:19 |
|
hwpplayer1 joined #minetest-dev |
| 14:34 |
|
SFENCE_arch joined #minetest-dev |
| 15:49 |
|
Desour joined #minetest-dev |
| 18:04 |
|
SFENCE joined #minetest-dev |
| 18:35 |
MTDiscord |
<josiah_wi> I have an idea to fix the benchmarks, but before I put it on the issue, someone tell me if this is worth considering. |
| 18:36 |
MTDiscord |
<josiah_wi> I propose saving the output from the unit tests and diffing it with the output from the benchmarks to extract only the benchmark output. |
| 18:37 |
MTDiscord |
<josiah_wi> Otherwise, the right way to solve the problem just in the context of Catch2 is probably to tag all the tests so we can specify which subset of tests to run. |
| 18:38 |
MTDiscord |
<josiah_wi> The root problem is that the typical use-case for unit tests is that you compile them separately (individually usually) from your application and benchmarks, so Catch2 isn't designed well for what we do. |
| 18:39 |
MTDiscord |
<josiah_wi> In retrospect this should have optimally come up before deciding to introduce Catch2 unit tests, but the way we discovered the problem existed was by running into it. |
| 18:40 |
sfan5 |
we could definitely just move it into a separate executable |
| 18:41 |
MTDiscord |
<josiah_wi> I imagined there was some reason we didn't do that already. Do we include tests and benchmarks in the binary for historical reasons? Does it have to do with being able to run tests on users platforms that we might not be able to test ourselves? |
| 18:42 |
sfan5 |
you could call it historical reasons since the unittests were always in the main executable |
| 18:42 |
sfan5 |
but actually |
| 18:43 |
sfan5 |
for the benchmark run, can't we just disable building the unit tests? |
| 18:43 |
MTDiscord |
<josiah_wi> Yes, totally. Is the benchmark run its own separate CI job? That makes the solution a piece of cake haha. |
| 18:44 |
MTDiscord |
<josiah_wi> The job will use less time and resources as well if it doesn't have to compile 50 extra test files for nothing. |
| 18:46 |
MTDiscord |
<josiah_wi> Also I see that #6987 happened; I don't see an issue suggesting splitting tests out of the main executable. Probably there are more important things to work on but that'd be in the same vein as not shipping devtest I think. |
| 18:46 |
ShadowBot |
https://github.com/minetest/minetest/issues/6987 -- Stop shipping devtest game in release builds? |
| 19:39 |
MTDiscord |
<luatic> Indeed it is. Here's the cake: https://github.com/luanti-org/benchmarks/pull/7. I would've committed directly but it seems I lack the permissions on this repo. I suppose @celeron55 has to fix that. |
| 19:39 |
MTDiscord |
<luatic> Yep, I think benchmarks and tests in separate executables would be nice, but aren't very important. |
| 19:42 |
MTDiscord |
<luatic> FWIW, I don't think the way it went is too bad. The "breakage" that happened here was not easy to expect and is of low significance. I think it's fine if something like this only gets caught later. |
| 19:43 |
MTDiscord |
<josiah_wi> Indeed, but I like retrospectives. |
| 19:43 |
MTDiscord |
<josiah_wi> Next time I set something like that up I will be aware of this potential problem so I learned something valuable. |
| 19:47 |
MTDiscord |
<celeron55> Fixed |
| 19:47 |
MTDiscord |
<celeron55> ...maybe? |
| 19:47 |
MTDiscord |
<celeron55> Are you in the Engine team? |
| 19:50 |
MTDiscord |
<celeron55> Yes you are, so it's fixed now |
| 19:51 |
MTDiscord |
<luatic> yep, just merged. hope it works. (this kind of CI is awkward to test) |
| 19:51 |
MTDiscord |
<luatic> not like there is anything to break |
| 19:54 |
MTDiscord |
<luatic> i suppose we'll see whether it worked next time we push to luanti |
| 19:55 |
MTDiscord |
<luatic> ah yeah @celeron55, while you're here, maybe irc.minetest.net -> irc.luanti.org? |
| 19:56 |
MTDiscord |
<luatic> should probably do that at the same time as the IRC channel rename, if we do that |
| 19:58 |
celeron55 |
yeah i definitely should try to get the irc channel rename done just about yesterday but the fact is i haven't |
| 21:49 |
|
YuGiOhJCJ joined #minetest-dev |
| 23:35 |
|
panwolfram joined #minetest-dev |
