Commit graph

68 commits

Author SHA1 Message Date
gdkchan
6e0799580f
Fix render target clear when sizes mismatch (#2994) 2022-01-11 20:15:17 +01:00
riperiperi
ef24c8983d
Fix adjacent 3d texture slices being detected as Incompatible Overlaps (#2993)
This fixes some regressions caused by #2971 which caused rendered 3D texture data to be lost for most slices. Fixes issues with Xenoblade 2's colour grading, probably a ton of other games.

This also removes the check from TextureCache, making it the tiniest bit smaller (any win is a win here).
2022-01-11 09:37:40 +01:00
riperiperi
cda659955c
Texture Sync, incompatible overlap handling, data flush improvements. (#2971)
* Initial test for texture sync

* WIP new texture flushing setup

* Improve rules for incompatible overlaps

Fixes a lot of issues with Unreal Engine games. Still a few minor issues (some caused by dma fast path?) Needs docs and cleanup.

* Cleanup, improvements

Improve rules for fast DMA

* Small tweak to group together flushes of overlapping handles.

* Fixes, flush overlapping texture data for ASTC and BC4/5 compressed textures.

Fixes the new Life is Strange game.

* Flush overlaps before init data, fix 3d texture size/overlap stuff

* Fix 3D Textures, faster single layer flush

Note: nosy people can no longer merge this with Vulkan. (unless they are nosy enough to implement the new backend methods)

* Remove unused method

* Minor cleanup

* More cleanup

* Use the More Fun and Hopefully No Driver Bugs method for getting compressed tex too

This one's for metro

* Address feedback, ASTC+ETC to FormatClass

* Change offset to use Span slice rather than IntPtr Add

* Fix this too
2022-01-09 13:28:48 -03:00
gdkchan
c05c8e09d4
Add support for the R4G4 texture format (#2956) 2021-12-30 17:10:54 +01:00
gdkchan
a87f7f2029
Fix DMA copy fast path line size when xCount < stride (#2942) 2021-12-26 13:05:26 -03:00
riperiperi
bc4e70b6fa
Move texture anisotropy check to SetInfo (#2843)
Rather than calculating this for every sampler, this PR calculates if a texture can force anisotropy when its info is set, and exposes the value via a public boolean.

This should help texture/sampler heavy games when anisotropic filtering is not Auto, like UE4 ones (or so i hear?). There is another cost where samplers are created twice when anisotropic filtering is enabled, but I'm not sure how relevant this one is.
2021-12-08 18:09:36 -03:00
gdkchan
ac4ec1a015
Account for negative strides on DMA copy (#2623)
* Account for negative strides on DMA copy

* Should account for non-zero Y
2021-09-11 22:54:18 +02:00
riperiperi
b0e410a828
Lift textures in the AutoDeleteCache for all modifications. (#2615)
* Lift textures in the AutoDeleteCache for all modifications.

Before, this would only apply to render targets and texture blit. Now it applies to image stores, the fast dma copy path and any other type of modification.

Image store always at least has one reference in the texture pool, so the function of the AutoDeleteCache keeping textures _alive_ is not useful, but a very important function for a while has been its use to flush textures in order of modification when they are dereferenced, so that their data is not lost.

Before, textures populated using image stores were being dereferenced and reloaded as garbage. Now, when these textures are dereferenced, their data will be put back into memory, and everything stays intact.

Fixes lighting breaking when switching levels in THPS1+2, and potentially some more UE4 games. I've tested a bunch more games for regressions and performance impact, but they all seem fine.

* Lift copy srcTexture so that it doesn't remain referenceless

* Perform lift before reference count change on unbind.

It's important to lift on unbind as that is the moment the texture was truly last modified, but definitely not after releasing every single reference.
2021-09-11 21:52:54 +02:00
riperiperi
15e7fe3ac9
Avoid deleting textures when their data does not overlap. (#2601)
* Avoid deleting textures when their data does not overlap.

It's possible that while two textures start and end addresses indicate an overlap, that the actual data contained within them is sparse due to a layer stride. One such possibility is array slices of a cubemap at different mip levels - they overlap on a whole, but the actual texture data fills the gaps between each other's layers rather than actually overlapping.

This fixes issues with UE4 games having incorrect lighting (solid white screen or really dark shadows). There are still remaining issues with games that use the 3D texture prebaked lighting, such as THPS1+2.

This PR also fixes a bug with TexturePool's resized texture handling where the base level in the descriptor was not considered.

* AllRegions granularity for 3d textures is now by level rather than by slice.

* Address feedback
2021-08-29 16:22:13 -03:00
riperiperi
bdc1f91a5b
Remove pool cache entries for incompatible overlapping textures (#2568)
This greatly reduces memory usage in games that aggressively reuse memory without removing dead textures from the pool, such as the Xenoblade games, UE3 games, and to a lesser extent, UE4/unity games.

This change stops memory usage from ballooning in xenoblade and some other games. It will also reduce texture view/dependency complexity in some games - for example in MK8D it will reduce the number of surface copies between lighting cubemaps generated for actors.

There shouldn't be any performance impact from doing this, though the deletion and creation of textures could be improved by improving the OpenGL texture storage cache, which is very simple and limited right now. This will be improved in future.

Another potential error has been fixed with the texture cache, which could prevent data loss when data is interchangably written to textures from both the GPU and CPU. It was possible that the dirty flag for a texture would be consumed without the data being synchronized on next use, due to the old overlap check. This check no longer consumes the dirty flag.

Please test a bunch of games to make sure they still work, and there are no performance regressions.
2021-08-20 17:52:09 -03:00
riperiperi
0a80a837cb
Use "Undesired" scale mode for certain textures rather than blacklisting (#2537)
* Use "Undesired" scale mode for certain textures rather than blacklisting

* Nit

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2021-08-11 22:44:51 +02:00
riperiperi
4b60371e64
Return mapped buffer pointer directly for flush, WriteableRegion for textures (#2494)
* Return mapped buffer pointer directly for flush, WriteableRegion for textures

A few changes here to generally improve performance, even for platforms not using the persistent buffer flush.

- Texture and buffer flush now return a ReadOnlySpan<byte>. It's guaranteed that this span is pinned in memory, but it will be overwritten on the next flush from that thread, so it is expected that the data is used before calling again.
- As a result, persistent mappings no longer copy to a new array - rather the persistent map is returned directly as a Span<>. A similar host array is used for the glGet flushes instead of allocating new arrays each time.
- Texture flushes now do their layout conversion into a WriteableRegion when the texture is not MultiRange, which allows the flush to happen directly into guest memory rather than into a temporary span, then copied over. This avoids another copy when doing layout conversion.

Overall, this saves 1 data copy for buffer flush, 1 copy for linear textures with matching source/target stride, and 2 copies for block textures or linear textures with mismatching strides.

* Fix tests

* Fix array pointer for Mesa/Intel path

* Address some feedback

* Update method for getting array pointer.
2021-07-19 19:10:54 -03:00
riperiperi
ca5ac37cd6
Flush buffers and texture data through a persistent mapped buffer. (#2481)
* Use persistent buffers to flush texture data

* Flush buffers via copy to persistent buffers.

* Log error when timing out, small refactoring.
2021-07-16 18:10:20 -03:00
gdkchan
fbb4019ed5
Initial support for separate GPU address spaces (#2394)
* Make GPU memory manager a member of GPU channel

* Move physical memory instance to the memory manager, and the caches to the physical memory

* PR feedback
2021-06-29 19:32:02 +02:00
gdkchan
a10b2c5ff2
Initial support for GPU channels (#2372)
* Ground work for separate GPU channels

* Rename TextureManager to TextureCache

* Decouple texture bindings management from the texture cache

* Rename BufferManager to BufferCache

* Decouple buffer bindings management from the buffer cache

* More comments and proper disposal

* PR feedback

* Force host state update on channel switch

* Typo

* PR feedback

* Missing using
2021-06-24 01:51:41 +02:00
riperiperi
79092310fa
Compare aligned size for largest mip level when considering sampler resize (#2306)
* Compare aligned size for largest mip level when considering sampler resize

When selecting a texture that's a view for a sampler resize, we should take care that resizing it doesn't change the aligned size of any larger mip levels.

This PR covers two cases:
- When creating a view of the texture, we check that the aligned size of the view shifted up to level 0 still matches the aligned size of the container. If it does not, a copy dependency is created rather than resizing.
- When searching for a texture for sampler, textures that do _not_ match our aligned size when both are shifted up by its base level are not considered an exact match, as resizing the found texture will cause the mip 0 aligned size to change. It will create a copy dependency view instead.

Fixes graphical errors and crashes (on flush) in various Unity games that use render-to-texture.

* Move shared code to its own method.
2021-05-24 17:35:26 +10:00
riperiperi
212e472c9f
Use copy dependencies for the Intel/AMD view format workaround (#2144)
* This might help AMD a bit

* Removal of old workaround.
2021-05-16 20:43:27 +02:00
gdkchan
f665e1b409
Hold reference for render targets in use (#2156) 2021-04-02 16:33:39 +02:00
riperiperi
8d36681bf1
Improve handling for unmapped GPU resources (#2083)
* Improve handling for unmapped GPU resources

- Fixed a memory tracking bug that would set protection on empty PTEs
- When a texture's memory is (partially) unmapped, all pool references are forcibly removed and the texture must be rediscovered to draw with it. This will also force the texture discovery to always compare the texture's range for a match.
- RegionHandles now know if they are unmapped, and automatically unset their dirty flag when unmapped.
- Partial texture sync now loads only the region of texture that has been modified. Unmapped memory tracking handles cause dirty flags for a texture group handle to be ignored.

This greatly improves the emulator's stability for newer UE4 games.

* Address feedback, fix MultiRange slice

Fixed an issue where the size of the multi-range slice would be miscalculated.

* Update Ryujinx.Memory/Range/MultiRange.cs (feedback)

Co-authored-by: Mary <thog@protonmail.com>

Co-authored-by: Mary <thog@protonmail.com>
2021-03-06 11:43:55 -03:00
riperiperi
b530f0e110
Texture Cache: "Texture Groups" and "Texture Dependencies" (#2001)
* Initial implementation (3d tex mips broken)

This works rather well for most games, just need to fix 3d texture mips.

* Cleanup

* Address feedback

* Copy Dependencies and various other fixes

* Fix layer/level offset for copy from view<->view.

* Remove dirty flag from dependency

The dirty flag behaviour is not needed - DeferredCopy is all we need.

* Fix tracking mip slices.

* Propagate granularity (fix astral chain)

* Address Feedback pt 1

* Save slice sizes as part of SizeInfo

* Fix nits

* Fix disposing multiple dependencies causing a crash

This list is obviously modified when removing dependencies, so create a copy of it.
2021-03-02 19:30:54 -03:00
gdkchan
67033ed8e0
Do not flush multisample textures (#1973) 2021-02-01 08:30:16 +01:00
gdkchan
b8353f5639
Enable parallel ASTC decoding by default (#1930) 2021-01-19 14:19:52 +11:00
gdkchan
c4f56c5704
Support for resources on non-contiguous GPU memory regions (#1905)
* Support for resources on non-contiguous GPU memory regions

* Implement MultiRange physical addresses, only used with a single range for now

* Actually use non-contiguous ranges

* GetPhysicalRegions fixes

* Documentation and remove Address property from TextureInfo

* Finish implementing GetWritableRegion

* Fix typo
2021-01-17 19:44:34 +01:00
riperiperi
24d316cc92
Extract texture Target from Info for quick access (#1774) 2020-12-03 20:34:27 +01:00
riperiperi
0108004691
Prefer truly perfect texture matches over fomat aliased ones (#1754) 2020-11-27 19:46:23 +01:00
riperiperi
9493cdfe55
Allow copy destination to have a different scale from source (#1711)
* Allow copy destination to have a different scale from source

Will result in more scaled copy destinations, but allows scaling in some games that copy textures to the output framebuffer.

* Support copying multiple levels/layers

Uses glFramebufferTextureLayer to copy multiple layers, copies levels individually (and scales the regions).

Remove CopyArrayScaled, since the backend copy handles it now.
2020-11-20 17:14:45 -03:00
gdkchan
7f536b5a15
Do not perform layout conversion on buffer texture flushes (#1729) 2020-11-18 22:17:40 +01:00
riperiperi
02872833b6
Size hints for copy regions and viewport dimensions to avoid data loss (#1686)
* Size hints for copy regions and viewport dimensions to avoid data loss

* Reword comment.

* Use info for the rule rather than calculating aligned size.

* Reorder min/max, remove spaces
2020-11-09 21:41:13 -03:00
gdkchan
11a7c99764
Support 3D BC4 and BC5 compressed textures (#1655)
* Support 3D BC4 and BC5 compressed textures

* PR feedback

* Fix some typos
2020-11-01 15:32:53 -03:00
gdkchan
423da5cc91
Scale texture resolution before sending to backend (#1646)
* Work

* Propagate scale factor to copy temp. Not really needed, just here for consistency

* PR feedback
2020-10-29 22:57:34 +01:00
gdkchan
cf0f0fc4e7
Improve the speed of redundant ASTC texture data updates (#1636) 2020-10-25 17:09:45 -03:00
gdkchan
2dcc6333f8
Fix image binding format (#1625)
* Fix image binding format

* XML doc
2020-10-20 19:03:20 -03:00
riperiperi
b4d8d893a4
Memory Read/Write Tracking using Region Handles (#1272)
* WIP Range Tracking

- Texture invalidation seems to have large problems
- Buffer/Pool invalidation may have problems
- Mirror memory tracking puts an additional `add` in compiled code, we likely just want to make HLE access slower if this is the final solution.
- Native project is in the messiest possible location.
- [HACK] JIT memory access always uses native "fast" path
- [HACK] Trying some things with texture invalidation and views.

It works :)

Still a few hacks, messy things, slow things

More work in progress stuff (also move to memory project)

Quite a bit faster now.
- Unmapping GPU VA and CPU VA will now correctly update write tracking regions, and invalidate textures for the former.
- The Virtual range list is now non-overlapping like the physical one.
- Fixed some bugs where regions could leak.
- Introduced a weird bug that I still need to track down (consistent invalid buffer in MK8 ribbon road)

Move some stuff.

I think we'll eventually just put the dll and so for this in a nuget package.

Fix rebase.

[WIP] MultiRegionHandle variable size ranges

- Avoid reprotecting regions that change often (needs some tweaking)
- There's still a bug in buffers, somehow.
- Might want different api for minimum granularity

Fix rebase issue

Commit everything needed for software only tracking.

Remove native components.

Remove more native stuff.

Cleanup

Use a separate window for the background context, update opentk. (fixes linux)

Some experimental changes

Should get things working up to scratch - still need to try some things with flush/modification and res scale.

Include address with the region action.

Initial work to make range tracking work

Still a ton of bugs

Fix some issues with the new stuff.

* Fix texture flush instability

There's still some weird behaviour, but it's much improved without this. (textures with cpu modified data were flushing over it)

* Find the destination texture for Buffer->Texture full copy

Greatly improves performance for nvdec videos (with range tracking)

* Further improve texture tracking

* Disable Memory Tracking for view parents

This is a temporary approach to better match behaviour on master (where invalidations would be soaked up by views, rather than trigger twice)

The assumption is that when views are created to a texture, they will cover all of its data anyways. Of course, this can easily be improved in future.

* Introduce some tracking tests.

WIP

* Complete base tests.

* Add more tests for multiregion, fix existing test.

* Cleanup Part 1

* Remove unnecessary code from memory tracking

* Fix some inconsistencies with 3D texture rule.

* Add dispose tests.

* Use a background thread for the background context.

Rather than setting and unsetting a context as current, doing the work on a dedicated thread with signals seems to be a bit faster.

Also nerf the multithreading test a bit.

* Copy to texture with matching alignment

This extends the copy to work for some videos with unusual size, such as tutorial videos in SMO. It will only occur if the destination texture already exists at XCount size.

* Track reads for buffer copies. Synchronize new buffers before copying overlaps.

* Remove old texture flushing mechanisms.

Range tracking all the way, baby.

* Wake the background thread when disposing.

Avoids a deadlock when games are closed.

* Address Feedback 1

* Separate TextureCopy instance for background thread

Also `BackgroundContextWorker.InBackground` for a more sensible idenfifier for if we're in a background thread.

* Add missing XML docs.

* Address Feedback

* Maybe I should start drinking coffee.

* Some more feedback.

* Remove flush warning, Refocus window after making background context
2020-10-16 17:18:35 -03:00
gdkchan
86412ed30a
Supper 2D array ASTC compressed texture formats decoding (#1593) 2020-10-02 11:22:23 +10:00
riperiperi
5dd6f41ff4
Make viewStorage still valid after view removal. (#1564) 2020-09-21 16:51:33 -03:00
riperiperi
5d69d9103e
Texture/Buffer Memory Management Improvements (#1408)
* Initial implementation. Still pending better valid-overlap handling,
disposed pool, compressed format flush fix.

* Very messy backend resource cache.

* Oops

* Dispose -> Release

* Improve Release/Dispose.

* More rule refinement.

* View compatibility levels as an enum - you can always know if a view is only copy compatible.

* General cleanup.

Use locking on the resource cache, as it is likely to be used by other threads in future.

* Rename resource cache to resource pool.

* Address some of the smaller nits.

* Fix regression with MK8 lens flare

Texture flushes done the old way should trigger memory tracking.

* Use TextureCreateInfo as a key.

It now implements IEquatable and generates a hashcode based on width/height.

* Fix size change for compressed+non-compressed view combos.

Before, this could set either the compressed or non compressed texture with a size with the wrong size, depending on which texture had its size changed. This caused exceptions when flushing the texture.

Now it correctly takes the block size into account, assuming that these textures are only related because a pixel in the non-compressed texture represents a block in the compressed one.

* Implement JD's suggestion for HashCode Combine

Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>

* Address feedback

* Address feedback.

Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>
2020-09-10 16:44:04 -03:00
gdkchan
bdfbcf4017
Fix regression on texture compatibility match checks (#1521) 2020-09-01 16:58:40 +10:00
sharmander
bc19114bb5
Fix: Issue #1475 Texture Compatibility Check methods need to be centralized (#1482)
* Texture Compatibility Check methods need to be centralized #1475

* Fix spacing

* Fix spacing

* Undo removal of .ToString()

* Move isPerfectMatch back to Texture.cs

Rename parameters in TextureCompatibility.cs for consistency

* Add switch from 1474 to TextureCompatibility as requested by mageven.

* Actually add TextureCompatibility changes to the PR (Add DeriveDepthFormat method)

* Alignment corrections + Derive method signature adjustment.

* Removed empty line as erquested

* Remove empty lines

* Remove blank lines, fix alignment

* Fix alignment

* Remove emtpy line
2020-08-31 21:06:27 -03:00
mageven
a33dc2f491
Improved Logger (#1292)
* Logger class changes only

Now compile-time checking is possible with the help of Nullable Value
types.

* Misc formatting

* Manual optimizations

PrintGuestLog
PrintGuestStackTrace
Surfaceflinger DequeueBuffer

* Reduce SendVibrationXX log level to Debug

* Add Notice log level

This level is always enabled and used to print system info, etc...
Also, rewrite LogColor to switch expression as colors are static

* Unify unhandled exception event handlers

* Print enabled LogLevels during init

* Re-add App Exit disposes in proper order

nit: switch case spacing

* Revert PrintGuestStackTrace to Info logs due to #1407

PrintGuestStackTrace is now called in some critical error handlers
so revert to old behavior as KThread isn't part of Guest.

* Batch replace Logger statements
2020-08-04 01:32:53 +02:00
gdkchan
2900dda633
Fix depth stencil formats copy by matching equivalent color formats (#1198) 2020-07-13 21:41:30 +10:00
riperiperi
484eb645ae
Implement Zero-Configuration Resolution Scaling (#1365)
* Initial implementation of Render Target Scaling

Works with most games I have. No GUI option right now, it is hardcoded.

Missing handling for texelFetch operation.

* Realtime Configuration, refactoring.

* texelFetch scaling on fragment shader (WIP)

* Improve Shader-Side changes.

* Fix potential crash when no color/depth bound

* Workaround random uses of textures in compute.

This was blacklisting textures in a few games despite causing no bugs. Will eventually add full support so this doesn't break anything.

* Fix scales oscillating when changing between non-native scales.

* Scaled textures on compute, cleanup, lazier uniform update.

* Cleanup.

* Fix stupidity

* Address Thog Feedback.

* Cover most of GDK's feedback (two comments remain)

* Fix bad rename

* Move IsDepthStencil to FormatExtensions, add docs.

* Fix default config, square texture detection.

* Three final fixes:

- Nearest copy when texture is integer format.
- Texture2D -> Texture3D copy correctly blacklists the texture before trying an unscaled copy (caused driver error)
- Discount small textures.

* Remove scale threshold.

Not needed right now - we'll see if we run into problems.

* All CPU modification blacklists scale.

* Fix comment.
2020-07-07 04:41:07 +02:00
gdkchan
b8eb6abecc
Refactor shader GPU state and memory access (#1203)
* Refactor shader GPU state and memory access

* Fix NVDEC project build

* Address PR feedback and add missing XML comments
2020-05-06 11:02:28 +10:00
gdkchan
f77694e4f7
Implement a new physical memory manager and replace DeviceMemory (#856)
* Implement a new physical memory manager and replace DeviceMemory

* Proper generic constraints

* Fix debug build

* Add memory tests

* New CPU memory manager and general code cleanup

* Remove host memory management from CPU project, use Ryujinx.Memory instead

* Fix tests

* Document exceptions on MemoryBlock

* Fix leak on unix memory allocation

* Proper disposal of some objects on tests

* Fix JitCache not being set as initialized

* GetRef without checks for 8-bits and 16-bits CAS

* Add MemoryBlock destructor

* Throw in separate method to improve codegen

* Address PR feedback

* QueryModified improvements

* Fix memory write tracking not marking all pages as modified in some cases

* Simplify MarkRegionAsModified

* Remove XML doc for ghost param

* Add back optimization to avoid useless buffer updates

* Add Ryujinx.Cpu project, move MemoryManager there and remove MemoryBlockWrapper

* Some nits

* Do not perform address translation when size is 0

* Address PR feedback and format NativeInterface class

* Remove ghost parameter description

* Update Ryujinx.Cpu to .NET Core 3.1

* Address PR feedback

* Fix build

* Return a well defined value for GetPhysicalAddress with invalid VA, and do not return unmapped ranges as modified

* Typo
2020-05-04 08:54:50 +10:00
gdkchan
34d19f381c
Fix texture level offset/size calculation when sparse tile width is > 1 (#1142)
* Fix texture level offset/size calculation when sparse tile width is > 1

* Sparse tile width affects layer size alignment aswell
2020-04-25 23:40:20 +10:00
gdkchan
3cb1fa0e85
Implement texture buffers (#1152)
* Implement texture buffers

* Throw NotSupportedException where appropriate
2020-04-25 23:02:18 +10:00
gdkchan
8e64984158
Support partial invalidation on texture access (#1000)
* Support partial invalidation on texture access

* Fix typo

* PR feedback

* Fix modified size clamping
2020-03-20 14:17:11 +11:00
riperiperi
6db16b4110
Only enumerate cached textures that are modified when flushing. (#918)
* Only enumarate cached textures that are modified when flushing, rather than all of them.

* Remove locking.

* Add missing clear.

* Remove texture from modified list when data is disposed.

In case the game does not call either flush method at any point.

* Add ReferenceEqualityComparer from jD for the HashSet
2020-02-07 08:49:26 +11:00
gdkchan
b8e3909d80 Add a GetSpan method to the memory manager and use it on GPU (#877) 2020-01-13 10:27:50 +11:00
gdkchan
a11f6f5235 Fix some spelling mistakes
Thanks to LDj3SNuD for spotting these
2020-01-09 02:13:00 +01:00
gdkchan
92703af555 Address PR feedback 2020-01-09 02:13:00 +01:00