Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

xrCore: rt_compressor.cpp: Use appropriate integer type #10

Merged
merged 1 commit into from
Aug 15, 2023
Merged

xrCore: rt_compressor.cpp: Use appropriate integer type #10

merged 1 commit into from
Aug 15, 2023

Conversation

clayne
Copy link
Contributor

@clayne clayne commented Aug 14, 2023

  • rtc_compress, rtc_decompress: Both of these functions pass a pointer to an out_size variable which lzo1x_1_compress and lzo1x_1_decompress dereference to store the value for the caller to use. The problem is that with a u32 type it's passing a pointer to a 4-byte integer but cast as an 8-byte integer hence the dereference and store is unsafe.
    *out_len = 0;
00000001405064F6  mov         rax,qword ptr [out_len]
00000001405064FE  mov         qword ptr [rsp+58h],rax
0000000140506503  mov         rcx,qword ptr [rsp+58h]
0000000140506508  call        __asan_store8 (014355B111h)
000000014050650D  mov         rax,qword ptr [out_len]
0000000140506515  mov         qword ptr [rax],0 <----- *qword*

Found with address sanitizer.
@clayne
xrCore: rt_compressor.cpp: Use appropriate integer type
a131fb8 
* rtc_compress, rtc_decompress: Both of these functions pass a pointer
  to an out_size variable which lzo1x_1_compress and lzo1x_1_decompress
  dereference to store the value for the caller to use. The problem is
  that with a u32 type it's passing a pointer to a 4-byte integer but
  cast as an 8-byte integer hence the dereference and store is unsafe.

    *out_len = 0;
00000001405064F6  mov         rax,qword ptr [out_len]
00000001405064FE  mov         qword ptr [rsp+58h],rax
0000000140506503  mov         rcx,qword ptr [rsp+58h]
0000000140506508  call        __asan_store8 (014355B111h)
000000014050650D  mov         rax,qword ptr [out_len]
0000000140506515  mov         qword ptr [rax],0 <----- *qword*

  Found with address sanitizer.
@themrdemonized themrdemonized merged commit fa77274 into themrdemonized:all-in-one-vs2022 Aug 15, 2023
themrdemonized added a commit that referenced this pull request Aug 15, 2023
@themrdemonized
Merge pull request #10 from clayne/1692030340-rt-compressor-stack-smash
fa5edb4 
xrCore: rt_compressor.cpp: Use appropriate integer type
(cherry picked from commit fa77274)
themrdemonized added a commit that referenced this pull request Aug 15, 2023
@themrdemonized
Merge pull request #10 from clayne/1692030340-rt-compressor-stack-smash
c6465df 
xrCore: rt_compressor.cpp: Use appropriate integer type
(cherry picked from commit fa77274)
(cherry picked from commit fa5edb4)
@clayne clayne deleted the 1692030340-rt-compressor-stack-smash branch August 19, 2023 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants
@clayne @themrdemonized