Commit graph

98 commits

Author SHA1 Message Date
riperiperi
9b7335a63b
Improve linear texture compatibility rules (#2099)
* Improve linear texture compatibility rules

Fixes an issue where small or width-aligned (rather than byte aligned) textures would fail to create a view of existing data. Creates a copy dependency as size change may be risky.

* Minor cleanup

* Remove Size Change for Copy Depenedencies

The copy to the target (potentially different sized) texture can properly deal with cropping by itself.

* Move StrideAlignment and GobAlignment into Constants
2021-03-19 02:17:38 +01:00
riperiperi
1623ab524f
Improve Buffer Textures and flush Image Stores (#2088)
* Improve Buffer Textures and flush Image Stores

Fixes a number of issues with buffer textures:

- Reworked Buffer Textures to create their buffers in the TextureManager, then bind them with the BufferManager later.
  - Fixes an issue where a buffer texture's buffer could be invalidated after it is bound, but before use.
- Fixed width unpacking for large buffer textures. The width is now 32-bit rather than 16.
- Force buffer textures to be rebound whenever any buffer is created, as using the handle id wasn't reliable, and the cost of binding isn't too high.

Fixes vertex explosions and flickering animations in UE4 games.

* Set ImageStore flag... for ImageStore.

* Check the offset and size.
2021-03-08 18:43:39 -03:00
riperiperi
8d36681bf1
Improve handling for unmapped GPU resources (#2083)
* Improve handling for unmapped GPU resources

- Fixed a memory tracking bug that would set protection on empty PTEs
- When a texture's memory is (partially) unmapped, all pool references are forcibly removed and the texture must be rediscovered to draw with it. This will also force the texture discovery to always compare the texture's range for a match.
- RegionHandles now know if they are unmapped, and automatically unset their dirty flag when unmapped.
- Partial texture sync now loads only the region of texture that has been modified. Unmapped memory tracking handles cause dirty flags for a texture group handle to be ignored.

This greatly improves the emulator's stability for newer UE4 games.

* Address feedback, fix MultiRange slice

Fixed an issue where the size of the multi-range slice would be miscalculated.

* Update Ryujinx.Memory/Range/MultiRange.cs (feedback)

Co-authored-by: Mary <thog@protonmail.com>

Co-authored-by: Mary <thog@protonmail.com>
2021-03-06 11:43:55 -03:00
riperiperi
b530f0e110
Texture Cache: "Texture Groups" and "Texture Dependencies" (#2001)
* Initial implementation (3d tex mips broken)

This works rather well for most games, just need to fix 3d texture mips.

* Cleanup

* Address feedback

* Copy Dependencies and various other fixes

* Fix layer/level offset for copy from view<->view.

* Remove dirty flag from dependency

The dirty flag behaviour is not needed - DeferredCopy is all we need.

* Fix tracking mip slices.

* Propagate granularity (fix astral chain)

* Address Feedback pt 1

* Save slice sizes as part of SizeInfo

* Fix nits

* Fix disposing multiple dependencies causing a crash

This list is obviously modified when removing dependencies, so create a copy of it.
2021-03-02 19:30:54 -03:00
gdkchan
7016d95eb1
Implement ETC2 (RGB) texture format (#2000)
* Implement ETC2 format

* Fix component counts for compressed formats
2021-02-08 10:23:56 +11:00
gdkchan
67033ed8e0
Do not flush multisample textures (#1973) 2021-02-01 08:30:16 +01:00
riperiperi
c30504e3b3
Use a descriptor cache for faster pool invalidation. (#1977)
* Use a descriptor cache for faster pool invalidation.

* Speed up comparison by casting to Vector256

Now we never need to worry about this ever again
2021-01-29 14:19:06 +11:00
gdkchan
f565b0e5a6
Match texture if the physical range is the same (#1934)
* Match texture if the physical range is the same

* XML docs and comments
2021-01-23 13:38:00 +01:00
gdkchan
b8353f5639
Enable parallel ASTC decoding by default (#1930) 2021-01-19 14:19:52 +11:00
gdkchan
03aab63e03
Fix out of range exception when a invalid base lod is used (#1931) 2021-01-19 14:04:38 +11:00
gdkchan
c4f56c5704
Support for resources on non-contiguous GPU memory regions (#1905)
* Support for resources on non-contiguous GPU memory regions

* Implement MultiRange physical addresses, only used with a single range for now

* Actually use non-contiguous ranges

* GetPhysicalRegions fixes

* Documentation and remove Address property from TextureInfo

* Finish implementing GetWritableRegion

* Fix typo
2021-01-17 19:44:34 +01:00
gdkchan
3bad321d2b
Fix mipmap base level being ignored for sampled textures and images (#1911)
* Fix mipmap base level being ignored for sampled textures and images

* Fix layer size and max level for textures

* Missing XML doc + reorder comments
2021-01-15 19:14:00 +01:00
Ac_K
73f6149bd6
gpu: Implement missing texture formats (#1867)
* gpu: Implement Etc2Rgba texture format

* Add more format

* Fix wrong pixel format
2021-01-05 06:02:49 +01:00
sharmander
06aa8a7578
GPU - Improve Memory Allocation (#1722)
* Implement TreeMap from scratch.

Begin implementation of MemoryBlockManager

* Implement GetFreePosition using MemoryBlocks

* Implementation of Memory Management using a Tree.

Still some issues to work around, but promising thus far.

* Resolved invalid mapping issue.

Performance appears promising.

* Add tick metrics

* Use the logger instead

* Use debug loggin instead of info.

* Remove unnecessary code. Add descriptions of added functions.

* Improve memory allocation even further. As well as improve speed of position fetching.

* Add TreeDictionary to Ryujinx Commons

Removed Unnecessary  Usigns

* Add a Performance Profiler + Improve ReserveFixed

* Begin transition to allocation in nvdrv

* Create singleton nvmemallocator

* Moved Allocation into Nv Related Files

As requested by gdkchan, any allocation of memory has been moved into the driver files.

Mapping remains in the GPU MemoryManager.

* Remove unnecessary usings

* Add missing descriptions

* Correct descriptions

* Fix formatting.

* Remove unnecessary whitespace

* Formatting / Convention Updates

* Changes / Fixes

Made syntax and convention changes as requested by gdkchan.

Fixed an issue where IsRegionUsed would return the wrong boolean.

Fixed an issue where GetFreePosition was asked for an address instead of a size.

* Undo commenting of Assert in shader cache

* Update Ryujinx.Common/Collections/TreeDictionary.cs

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

* Resolved many suggestions

* Implement Improved TreeDictionary

Based off of Pseudo code and custom implementations.

* Rename _set to _dictionary

* Remove unused code

* Remove unused code.

* Remove unnecessary MapLow function.

* Resolve data-structure based issues

* Make adjustments to memory management.

Deactive de-allocation for now, it causes more harm than good.

* Minor refactorings + Re-implement deallocation

Also cleaned up unnecessary code.

* Add Tests for TreeDictionary

* Update data structure to properly balance the tree

* Experimental Implementation:

1. Reduce Time to Next Node to O(1) Runtime
2. Reduce While Loop Ct To 2 (In Most Cases)

* Address issues w/ Deallocating Memory

* Final Build

+ Fully Implement Dictionary Interface for new Data Structure
+ Cover All Memory Allocation Edge Cases, particularly w/ Games that De-Allocate a lot.

* Minor Corrections

Give TreeDictionary its own count (do not depend on inner dictionary)

Properly remove adjacent allocations

* Add AsList

* Fix bug where internal dictionary wasn't being updated w/ new node for overwritten key.

* Address comments in review.

* Fix issue where block wouldn't break out (Fixes UE4 issues)

* Update descriptions

* Update descriptions

* Reduce Node visibility to protect TreeDictionary Integrity + Remove usage of struct.

* Update tests to use new TreeDictionary implementation.

* Remove usage of dictionary in TreeDictionary

* Refactoring / Renaming

* Remove unneeded memoryblock class.

* Add space for while

* Add space for if

* Formatting / descriptions

* Clarified some descriptions

* Reduce visibility of memory allocator

* Edit method names to make more sense as memory blocks are no longer in use.

* Make names consistent.

* Protect against npe when sucessorof is called against keys that don't exist. (Not in use by memory manager, this is for other prs that might use this data structure)

* Possible edge-case resolve

* Update Ryujinx.Common/Collections/TreeDictionary.cs

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

* Update Ryujinx.HLE/HOS/Services/Nv/NvMemoryAllocator.cs

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

* Reduce # of unnecessary duplicate variables / Reduce visibility of variables only internally used.

* Rename count to _count

* Update Description of Add method.

* Fix copypasta

* Address comments

* Address comments

* Remove whitespace

* Address comments, condense variables.

* Consolidate vars

* Fix whitespace.

* Nit

* Fix exception msg

* Fix arrayIndex check

* Fix arrayIndex check + indexer

* Remove whitespace from cast

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2020-12-09 19:26:05 -03:00
riperiperi
24d316cc92
Extract texture Target from Info for quick access (#1774) 2020-12-03 20:34:27 +01:00
riperiperi
2c39a4f15d
Cache delegate for QueryModified, use regular multi handle. (#1771) 2020-12-03 19:34:32 +01:00
riperiperi
0108004691
Prefer truly perfect texture matches over fomat aliased ones (#1754) 2020-11-27 19:46:23 +01:00
riperiperi
88633f4bc2
Blacklist very textures with a very small width or height from scaling (#1753) 2020-11-27 19:23:30 +01:00
riperiperi
9493cdfe55
Allow copy destination to have a different scale from source (#1711)
* Allow copy destination to have a different scale from source

Will result in more scaled copy destinations, but allows scaling in some games that copy textures to the output framebuffer.

* Support copying multiple levels/layers

Uses glFramebufferTextureLayer to copy multiple layers, copies levels individually (and scales the regions).

Remove CopyArrayScaled, since the backend copy handles it now.
2020-11-20 17:14:45 -03:00
gdkchan
7f536b5a15
Do not perform layout conversion on buffer texture flushes (#1729) 2020-11-18 22:17:40 +01:00
Mary
383c039037
shader cache: Fix invalid virtual address clean up (#1717)
* shader cache: Fix invalid virtual address clean up

This fix an issue causing the virtual address of texture descriptors to
not be cleaned up when caching and instead cleaning texture format and swizzle.

This should fix duplicate high duplication in the cache for certain
games and possible texture corruption issues.

**THIS WILL INVALIDATE ALL SHADER CACHE LEVELS CONSIDERING THE NATURE OF THE ISSUE**

* shader cache: Address gdk's comment
2020-11-17 22:20:17 +01:00
Mary
48f6570557
Salieri: shader cache (#1701)
Here come Salieri, my implementation of a disk shader cache!

"I'm sure you know why I named it that."
"It doesn't really mean anything."

This implementation collects shaders at runtime and cache them to be later compiled when starting a game.
2020-11-13 00:15:34 +01:00
riperiperi
02872833b6
Size hints for copy regions and viewport dimensions to avoid data loss (#1686)
* Size hints for copy regions and viewport dimensions to avoid data loss

* Reword comment.

* Use info for the rule rather than calculating aligned size.

* Reorder min/max, remove spaces
2020-11-09 21:41:13 -03:00
gdkchan
934a78005e
Simplify logic for bindless texture handling (#1667)
* Simplify logic for bindless texture handling

* Nits
2020-11-09 19:35:04 -03:00
gdkchan
8d168574eb
Use explicit buffer and texture bindings on shaders (#1666)
* Use explicit buffer and texture bindings on shaders

* More XML docs and other nits
2020-11-08 12:10:00 +01:00
riperiperi
a16f201a6f
Do not align sizes for buffer texture targets. (#1671)
This should fix a random crash in Hyrule Warriors, and potentially other games that use buffer textures.
2020-11-06 18:45:30 +01:00
gdkchan
a89b81a812
Separate zeta from color formats (#1647) 2020-11-05 23:50:34 +01:00
riperiperi
4c6feb652f
Add seamless cubemap flag in sampler parameters. (#1658)
* Add seamless cubemap flag in sampler parameters.

* Check for the extension
2020-11-02 17:03:06 -03:00
riperiperi
e1da7df207
Support res scale on images, correctly blacklist for SUST, move logic out of backend. (#1657)
* Support res scale on images, correctly blacklist for SUST, move logic
out of backend.

* Fix Typo
2020-11-02 16:53:23 -03:00
gdkchan
11a7c99764
Support 3D BC4 and BC5 compressed textures (#1655)
* Support 3D BC4 and BC5 compressed textures

* PR feedback

* Fix some typos
2020-11-01 15:32:53 -03:00
gdkchan
876fa656f6
Remove unused texture and sampler pool invalidation code (#1648) 2020-11-01 15:17:29 -03:00
gdkchan
423da5cc91
Scale texture resolution before sending to backend (#1646)
* Work

* Propagate scale factor to copy temp. Not really needed, just here for consistency

* PR feedback
2020-10-29 22:57:34 +01:00
gdkchan
cf0f0fc4e7
Improve the speed of redundant ASTC texture data updates (#1636) 2020-10-25 17:09:45 -03:00
gdkchan
efa77a2415
Add missing null check on image binding (#1632) 2020-10-21 14:06:13 +02:00
gdkchan
2dcc6333f8
Fix image binding format (#1625)
* Fix image binding format

* XML doc
2020-10-20 19:03:20 -03:00
riperiperi
08332bdc04
Ensure storage is set for Buffer Textures when binding an Image. (#1627) 2020-10-20 18:56:23 -03:00
riperiperi
b4d8d893a4
Memory Read/Write Tracking using Region Handles (#1272)
* WIP Range Tracking

- Texture invalidation seems to have large problems
- Buffer/Pool invalidation may have problems
- Mirror memory tracking puts an additional `add` in compiled code, we likely just want to make HLE access slower if this is the final solution.
- Native project is in the messiest possible location.
- [HACK] JIT memory access always uses native "fast" path
- [HACK] Trying some things with texture invalidation and views.

It works :)

Still a few hacks, messy things, slow things

More work in progress stuff (also move to memory project)

Quite a bit faster now.
- Unmapping GPU VA and CPU VA will now correctly update write tracking regions, and invalidate textures for the former.
- The Virtual range list is now non-overlapping like the physical one.
- Fixed some bugs where regions could leak.
- Introduced a weird bug that I still need to track down (consistent invalid buffer in MK8 ribbon road)

Move some stuff.

I think we'll eventually just put the dll and so for this in a nuget package.

Fix rebase.

[WIP] MultiRegionHandle variable size ranges

- Avoid reprotecting regions that change often (needs some tweaking)
- There's still a bug in buffers, somehow.
- Might want different api for minimum granularity

Fix rebase issue

Commit everything needed for software only tracking.

Remove native components.

Remove more native stuff.

Cleanup

Use a separate window for the background context, update opentk. (fixes linux)

Some experimental changes

Should get things working up to scratch - still need to try some things with flush/modification and res scale.

Include address with the region action.

Initial work to make range tracking work

Still a ton of bugs

Fix some issues with the new stuff.

* Fix texture flush instability

There's still some weird behaviour, but it's much improved without this. (textures with cpu modified data were flushing over it)

* Find the destination texture for Buffer->Texture full copy

Greatly improves performance for nvdec videos (with range tracking)

* Further improve texture tracking

* Disable Memory Tracking for view parents

This is a temporary approach to better match behaviour on master (where invalidations would be soaked up by views, rather than trigger twice)

The assumption is that when views are created to a texture, they will cover all of its data anyways. Of course, this can easily be improved in future.

* Introduce some tracking tests.

WIP

* Complete base tests.

* Add more tests for multiregion, fix existing test.

* Cleanup Part 1

* Remove unnecessary code from memory tracking

* Fix some inconsistencies with 3D texture rule.

* Add dispose tests.

* Use a background thread for the background context.

Rather than setting and unsetting a context as current, doing the work on a dedicated thread with signals seems to be a bit faster.

Also nerf the multithreading test a bit.

* Copy to texture with matching alignment

This extends the copy to work for some videos with unusual size, such as tutorial videos in SMO. It will only occur if the destination texture already exists at XCount size.

* Track reads for buffer copies. Synchronize new buffers before copying overlaps.

* Remove old texture flushing mechanisms.

Range tracking all the way, baby.

* Wake the background thread when disposing.

Avoids a deadlock when games are closed.

* Address Feedback 1

* Separate TextureCopy instance for background thread

Also `BackgroundContextWorker.InBackground` for a more sensible idenfifier for if we're in a background thread.

* Add missing XML docs.

* Address Feedback

* Maybe I should start drinking coffee.

* Some more feedback.

* Remove flush warning, Refocus window after making background context
2020-10-16 17:18:35 -03:00
gdkchan
86412ed30a
Supper 2D array ASTC compressed texture formats decoding (#1593) 2020-10-02 11:22:23 +10:00
gdkchan
1560f236da
Convert 1D texture targets to 2D (#1584)
* Convert 1D texture targets to 2D

* Fix typo

* Simplify some code

* Should mask that too

* Consistency
2020-09-29 22:28:50 +02:00
riperiperi
f89b754abb
Always set new texture data for textures initialized by a copy. (#1576) 2020-09-27 09:37:45 +10:00
gdkchan
bd28ce90e6
Implement small indexed draws and other fixes to make guest Vulkan work (#1558) 2020-09-24 09:48:34 +10:00
riperiperi
5dd6f41ff4
Make viewStorage still valid after view removal. (#1564) 2020-09-21 16:51:33 -03:00
riperiperi
3d055da5fc
Allow swizzles to match with "undefined" components (#1538)
* Add swizzle matching rules.

Improves rules which try to match incompatible formats as perfect, such as D32 float -> R32 float.

Remove Format.HasOneComponent, since this information is now available via the FormatInfo struct.

* Fix this rule.

* Update component counts for depth formats.
2020-09-11 09:48:48 +10:00
riperiperi
5d69d9103e
Texture/Buffer Memory Management Improvements (#1408)
* Initial implementation. Still pending better valid-overlap handling,
disposed pool, compressed format flush fix.

* Very messy backend resource cache.

* Oops

* Dispose -> Release

* Improve Release/Dispose.

* More rule refinement.

* View compatibility levels as an enum - you can always know if a view is only copy compatible.

* General cleanup.

Use locking on the resource cache, as it is likely to be used by other threads in future.

* Rename resource cache to resource pool.

* Address some of the smaller nits.

* Fix regression with MK8 lens flare

Texture flushes done the old way should trigger memory tracking.

* Use TextureCreateInfo as a key.

It now implements IEquatable and generates a hashcode based on width/height.

* Fix size change for compressed+non-compressed view combos.

Before, this could set either the compressed or non compressed texture with a size with the wrong size, depending on which texture had its size changed. This caused exceptions when flushing the texture.

Now it correctly takes the block size into account, assuming that these textures are only related because a pixel in the non-compressed texture represents a block in the compressed one.

* Implement JD's suggestion for HashCode Combine

Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>

* Address feedback

* Address feedback.

Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>
2020-09-10 16:44:04 -03:00
gdkchan
bdfbcf4017
Fix regression on texture compatibility match checks (#1521) 2020-09-01 16:58:40 +10:00
sharmander
bc19114bb5
Fix: Issue #1475 Texture Compatibility Check methods need to be centralized (#1482)
* Texture Compatibility Check methods need to be centralized #1475

* Fix spacing

* Fix spacing

* Undo removal of .ToString()

* Move isPerfectMatch back to Texture.cs

Rename parameters in TextureCompatibility.cs for consistency

* Add switch from 1474 to TextureCompatibility as requested by mageven.

* Actually add TextureCompatibility changes to the PR (Add DeriveDepthFormat method)

* Alignment corrections + Derive method signature adjustment.

* Removed empty line as erquested

* Remove empty lines

* Remove blank lines, fix alignment

* Fix alignment

* Remove emtpy line
2020-08-31 21:06:27 -03:00
gdkchan
09341dc11d
Fix off by one error in pages count calculation on GPU pool (#1511) 2020-08-29 16:42:34 -03:00
mageven
a33dc2f491
Improved Logger (#1292)
* Logger class changes only

Now compile-time checking is possible with the help of Nullable Value
types.

* Misc formatting

* Manual optimizations

PrintGuestLog
PrintGuestStackTrace
Surfaceflinger DequeueBuffer

* Reduce SendVibrationXX log level to Debug

* Add Notice log level

This level is always enabled and used to print system info, etc...
Also, rewrite LogColor to switch expression as colors are static

* Unify unhandled exception event handlers

* Print enabled LogLevels during init

* Re-add App Exit disposes in proper order

nit: switch case spacing

* Revert PrintGuestStackTrace to Info logs due to #1407

PrintGuestStackTrace is now called in some critical error handlers
so revert to old behavior as KThread isn't part of Guest.

* Batch replace Logger statements
2020-08-04 01:32:53 +02:00
gdkchan
2900dda633
Fix depth stencil formats copy by matching equivalent color formats (#1198) 2020-07-13 21:41:30 +10:00
gdkchan
4d02a2d2c0
New NVDEC and VIC implementation (#1384)
* Initial NVDEC and VIC implementation

* Update FFmpeg.AutoGen to 4.3.0

* Add nvdec dependencies for Windows

* Unify some VP9 structures

* Rename VP9 structure fields

* Improvements to Video API

* XML docs for Common.Memory

* Remove now unused or redundant overloads from MemoryAccessor

* NVDEC UV surface read/write scalar paths

* Add FIXME comments about hacky things/stuff that will need to be fixed in the future

* Cleaned up VP9 memory allocation

* Remove some debug logs

* Rename some VP9 structs

* Remove unused struct

* No need to compile Ryujinx.Graphics.Host1x with unsafe anymore

* Name AsyncWorkQueue threads to make debugging easier

* Make Vp9PictureInfo a ref struct

* LayoutConverter no longer needs the depth argument (broken by rebase)

* Pooling of VP9 buffers, plus fix a memory leak on VP9

* Really wish VS could rename projects properly...

* Address feedback

* Remove using

* Catch OperationCanceledException

* Add licensing informations

* Add THIRDPARTY.md to release too

Co-authored-by: Thog <me@thog.eu>
2020-07-12 05:07:01 +02:00