Revisiting Sameboy for GB/GBC core

[TiKevin83]

This is a follow up to #1282.

EZGames and I are looking to resync TASes for the broader GB/GBC library. We've seen only minor improvements to syncing Mickey's Dangerous chase on GBP with the improvements MrWint made to Gambatte for Pokemon Crystal, we now get past the first enemy and die on the 2nd. GBHawk is still way off, doesn't even sync to console through the intro.

Seperately, DrD2K9 also messaged me about syncing Donkey Kong '94, and that game also does not appear to play near console on either Gambatte or GBHawk.

Gambatte is simply too tuned for Pokemon with little attention to other franchises.
GBHawk from what I gathered in the last post with Alyosha only really accounted for GB tests when there are numerous GBC only timing quirks. GBHawk also runs almost an order of magnitude slower than Gambatte, which is a significant factor for TASing. The inaccuracies in GBHawk were highlighted in the last issue and no progress has been made on them.

These incremental issues could be avoided entirely by using SameBoy as a GB core across all of GB, GBC, and SGB. SameBoy has constant active development and rigorous testing from a whole community of GB emu devs. It has no known timing inaccuracies on any game; any desync between the SameBoy core and console playback would be down to the game relying on initial SRAM state and being unsyncable, or a potentially revolutionary discovery about the GameBoy hardware. These cases can only be properly identified at this rate if SameBoy support is expanded.

[DrD2k9]

Specifically for Donkey Kong (aka DK'94): GBHawk and Gambatte cores emulate drastically differently in regards to loading times. This obviously affects timing as well as things like RNG. GBHawk seems to run the game well enough, but certain sounds don't play correctly (for example, When starting the mid-world DK fight at stage 1-4 the stage begins before the stage-intro SFX completes playing.).

If a SameBoy core would be more accurate to actual hardware, it would be better for TASing.

[YoshiRulz]

Do you know how up-to-date our core is to upstream?

[TiKevin83]

Yoshi I'm not sure what you're referring to as "our core."

[alyosha-tas]

All the tests you sent me pass for both GB and GBC mode in GBHawk, so certainly progress has been made. But yeah I haven’t worked on it in a while .

But anyway the sameboy core is quite out of date by now compared to upstream. And, like any other waterbox core , requires natt to update it. Natt seems to be not very active if late so this probably won’t happen very fast. Unless someone can work through natts toolxhain , which I’m not aware anyone has been successful at so far.

You might need to sort this out with Adelikat and zeromus , but I would suggest reimplementing sameboy as a ported core similar to Gambatte that you can more easily keep maintained .

[YoshiRulz]

@TiKevin83 our core's source is here. As alyosha said, waterboxed cores aren't getting enough maintenance. It'd be great if we could replace it all with a submodule to LIJI32/SameBoy.

[TiKevin83]

Yeah the Bizhawk SameBoy core is both not up to date enough for what I'm describing and only tied in for SGB. It sounds like porting SameBoy and dropping the waterboxed version is worth further investigation.

Alyosha, I can't seem to get any of the PPU tests in mooneye-gb/acceptance to run on GBHawk as GBC. There are precompiled binaries of the testroms available here. I can break that off into a separate issue if desired.

https://gekkio.fi/files/mooneye-gb/latest/

[alyosha-tas]

GB -> settings -> sync settings -> console mode -> GBC.
Also make sure timer Div initial time is 8.

Then reboot the core.

I tested Mickey's Dangerous chase just now and there are some definite differences. In particular some PPU IRQs don't fire in GBHawk like they do in gambatte, I'll look into it.

But I resynced a gambatte movie of it in GBHawk (accounting for changes in frame timing) and it definitely made it through the menues in the same way.

EDIT: Also the gambatte core doesn't behave correctly for many of the GBC mode tests (it still behaves in GB mode)

EDIT: Also are you making the Mickey movie from GBHawk in the correct console mode?

[TiKevin83]

With those sync settings when running the intr_2_mode3_timing test I get a blank screen on GBHawk 2.3.2 where on SameBoy I get a printout of registers with assertions D: OK and E: OK. I also get a blank screen on intr 2 mode0 timing,, stat irq blocking, and stat lyc onoff. I think something's going wrong unrelated to individual test results because I can't seem to get some of the basic tests to run, just blank screen after booting.

I'll have to test sync for GBHawk Mickey's Dangerous Chase later.

[alyosha-tas]

All the tests work fine for me on dev build, is it black screen, or is it booting up the GBC logo and then going blank screen?

[EZGames69]

There’s also the issue of loading screens being incredibly fast compared to gambette. I’d even go as far as to say they’re VBA levels of inaccurate.

For Mickey’s Chase specifically on Gambette, there would be something like 6-7 lag frames for each loading transition. But for GBHawk it only has one frame for loading. I made a gif comparing this here: https://gfycat.com/gifs/detail/foolhardyhomelyalligatorgar

[alyosha-tas]

Whenever the screen is off, GBHawk treats it as one frame. It's not inaccurate, just compressed for clean input.

[TiKevin83]

Booting up the GBC logo then going to blank screen. And I triple checked that I'm on GBHawk and not Gambatte.

[MemoryTAS]

Whenever the screen is off, GBHawk treats it as one frame. It's not inaccurate, just compressed for clean input.

This doesn't sound much better to me than VBA treating multiple lag frames in a row as one frame. I don't see how this is necessary.

[alyosha-tas]

Booting up the GBC logo then going to blank screen. And I triple checked that I'm on GBHawk and not Gambatte.

Alright I’ll look into it tomorrow

[LIJI32]

I’m willing to help with the integration, in case you need additional exported APIs in SameBoy. This will make it easier to upgrade the core when SameBoy updates.

[alyosha-tas]

Alright the test roms should display correctly in the dev build now. They were running correctly, just not displaying since I had some issues with the GBC compatibility register at FF4C. I need to take a second look at what bit turns on what stuff.

[nattthebear]

One thing I'm not quite following here about removing the waterbox core: As far as I remember, the waterboxed Sameboy in Bizhawk is only used as an SGB alternate. The core of course fully supports plain GB and CGB, but the romloader was never wired to it in the frontend. But these complaints in accuracy are about Bizhawk's GB and CGB support.

If the goal is better GB and CGB support, and a traditionally integrated Sameboy core is the best way to do that, then sure, go ahead, but it doesn't make sense to remove the waterbox core unless the SGB part of it is also wired in.

[TiKevin83]

I reran the mooneye-gb test suite against GBHawk and identified regressions in the following tests:
oam dma start
sources dmgABCmgbS
lcdon timing dmgABCmgbS
stat irq blocking
stat lyc onoff
I would be interested to see if it has improved in the wilbertpol tests.

Based on this I continue to believe the best way to guarantee emulation accuracy across the GB/GBC library is to use SameBoy as a core for TASing. This is not a dig on any of the work that has been done on GBHawk, just an observation on its current state.

[alyosha-tas]

All those tests work correctly for me. What build are you using? Try the dev build, Although they work fine in 2.4 as well.

[TiKevin83]

I was testing Release 2.4, but I think I actually used Gambatte on accident instead of GBHawk.
Those tests passed on GBHawk! Do you know if all the wilbertpol GPU tests are passing yet? I'd like to try a few more resyncs if so

[alyosha-tas]

It passes all of them (the wilbert pol tests) in the archive I have. I don't remember if it's up to date or not, but all the sprite timing, scroll timing, and LY timing tests are in there and pass / fail as they should for GB / GBC.

Also I haven't tested double speed mode at all except to verify games work, that is probably where most of the issues currently lie. But for anything that uses GB games or only single speed mode, I'm not aware of any issues except that obscure DMA glitch, which isn't relevant anyway (unless someone makes a homebrew purposely to exploit it.)

[LIJI32]

Considering SameBoy has built-in SGB support now, I believe removing the current core (which is really outdated, using both the old PPU and old APU implementation of SameBoy) and re-adding SameBoy from master would be easier to do, with the added bonus of having DMG and CGB support. Other than adding SGB support, were any changes required for the core? And are any core changes required to make SameBoy feature-equivalent to Gambatte? I'd love to have SameBoy integrated with minimal-to-zero core changes so it can be easily updated without effort, and I'd add/integrate any missing API to master if it helps. How much effort would it take to re-add SameBoy as a standalone core, and how can I help making it happen?

[nattthebear]

We never supported the DMG or CGB capabilities of SameBoy in BizHawk for political reasons; maybe someone else can chime in.

As far as SGB support goes; does it support games that use the SNES audio features? That's one capability that was unique to our core (among HLE SGB cores) when I last checked some time ago; and I'd hate to lose it.

I am very enthusiastic about SameBoy as a first class regularly updated core. It was a pleasure to work with when I was porting and adding SGB support, and I'd be happy to see more of it.

Unfortunately, I don't have much time to devote to BizHawk these days, so we'd need a porting champion.

[LIJI32]

No, SNES audio isn't supported, but I do consider some form of very high level emulation of the SNES based sound effects, but not music. (Using built in samples and/or synthesis of them, to avoid needing the SGB BIOS). How is it done in the BizHawk port of SameBoy?

[nattthebear]

I dropped in blargg's snes_spc library. It's then a reasonably simple matter to respond to commands 8 and 9 in the same way that the SNES side of a real SGB would, without needing more parts emulated.

[LIJI32]

Interesting. Considering snes_spc is LGPL it shouldn't be a problem to integrate it into SameBoy while keeping the MIT license.

[nattthebear]

Leaving a note here for myself about some changes that I made to our sameboy port that would need to be revisited. None of these are major issues, but I'll need a punch list of things to look at.

disassembler and debugger were removed entirely
savestate code was excised somewhat
advance for N cycles was added (as opposed to advance for 1 frame)
UD+LR blocking in core was removed (gb only)
large parts of timing.c/h related to frontend notions of time were removed
blip buffer was added, excising some of the internal audio code and replacing it with a sample callback
key input changed up a bit, and instead of latching at vblank we latch whenever we hawk
input callback + lag flag
scanline callback
saveram hookup
rtc hookup

[LIJI32]

A few notes:

disassembler and debugger were removed entirely
large parts of timing.c/h related to frontend notions of time were removed

A bit more recent versions of the SameBoy Core allow disabling these parts using flags and not compiling debugger.c and sm83_disassembler.c

advance for N cycles was added (as opposed to advance for 1 frame)

GB_run always existed, which advanced in opcode quantities.

blip buffer was added, excising some of the internal audio code and replacing it with a sample callback

Recent SameBoy versions (0.12 IIRC) use sample callbacks as well

key input changed up a bit, and instead of latching at vblank we latch whenever we hawk

Same, this is already done in recent versions too (The input isn't latched anymore, you can just call the set_input function whenever you want, and it applies instantly)

So these should reduce the number of changes required. Other than that, I'd love to merge other changes to the core into mainstream so future updates to the core are easier. This was done with bsnes, and updating bsnes from SameBoy 0.12.x to 0.13.x took mere minutes.

[nattthebear]

I believe the problem with both cycle advance and audio was related to mixing SGB and GB audio. I had to hack around some things to get me enough timing information to do that. I think resolving LIJI32/SameBoy#264 (which may not be easy, as I haven't done much work with it yet) completely will take care of that.

[red031000]

can I bump this? having an up-to-date sameboy for GB/GBC would be really beneficial, I'm willing to do the work myself if I can figure out how to do so

[YoshiRulz]

No submodule yet, but I've made a fork which at least gets our changes in the commit tree. The next step should be splitting the one monolith commit into several, then we can start thinking about rebasing.

[YoshiRulz]

We've removed the Sameboy core, but I'll leave the fork up in case anyone wants to look through it for patches to send upstream.

[alyosha-tas]

A lot of this thread is now well outdated, so I think this issue can be closed. In fact, with a few more updates Gambatte will obsolete GBHawk as well, then we'll have just one core that does it all, which I think is the best case scenario.