Struggling with my x265 settings

[Hostile_18]

I'm getting really good results with these settings now. The bit rate limits are kind of arbitrary according to chat gpt, so if they need tweaking please let me know (roughly min 40% of source, max 75% of source). I'll experiment with SAO as well, but all my main problems are sorted.

libx265 -pix_fmt yuv420p10le -crf 18 -preset slow -x265-params aq-mode=1:no-open-gop=1:high-tier=1:level-idc=4.1:bframes=6:subme=4:rc-lookahead=120bratio=1.25:ctu=32:merange=26:deblock=-3,-3:vbv-maxrate=20000:vbv-bufsize=40000:

Regarding audio I'm converting the lossless tracks to EAC3, is there a bit rate where it will be virtually no difference? Currently set to 960kbps (not each channel).

[Z2697]

x265 SAO + grain = no

vbv-maxrate and vbv-bufsize don't mean "min/max".
Think encoding as draining water (bits) from a bucket, the size of the bucket is the vbv-bufsize, the speed the bucket gets refilled is vbv-maxrate.
(This is why a bitrate overshoot is called vbv underflow)

Using Opus (libopus) for audio would be more efficient (unless you are making E-AC-3 JOC).

[GeoffreyA]

Regarding audio, I like Apple AAC-LC (QAAC). TVBR91 or 109 will give excellent results.

[Hostile_18]

Originally Posted by Z2697

x265 SAO + grain = no

vbv-maxrate and vbv-bufsize don't mean "min/max".
Think encoding as draining water (bits) from a bucket, the size of the bucket is the vbv-bufsize, the speed the bucket gets refilled is vbv-maxrate.
(This is why a bitrate overshoot is called vbv underflow)

Using Opus (libopus) for audio would be more efficient (unless you are making E-AC-3 JOC).

Ah I see, I mean it has left most films unaffected but the top end its reduced the file size, while showing no problems to my eyes with the quality. Do those values chat gpt came up with seem reasonable?

I was going to say Opus doesn't work with VLC player, but I've just seen an update for the software and now its working. What bit rate for Opus would be good where only an audiophile might be able to hear a difference? (compared to trueHD and DTS).

Edit: Just read Opus isn't supported well on LG OS TV's, I'll have to double check on that.

[Hostile_18]

I went with Opus in the end, everything can direct play no issues. My settings below are doing really well. Roughly about 2 and a half hours per disc. Someone on reddit thought I might be doing alot of complex maths and not using it with my settings, so was recommending faster presets? When I've looked into it compared to the medium preset the only setting I've got that adds a lot of time is "rect" If I turn it off one 60 second test video goes from 2.16 to encode to 1.30, with my slow preset settings below. Is Rect essential to my video quality or can I safely turn it off and save a lot of time? File size was roughly the same.

Star to Hex saved 8 seconds per 1 min test video, so not as dramatic, but again not to sure the visual cost.

libx265 -pix_fmt yuv420p10le -crf 17 -preset slow -x265-params aq-mode=1:no-open-gop=1:high-tier=1:level-idc=4.1:bframes=6:subme=4:rc-lookahead=120bratio=1.25:ctu=32:merange=26:deblock=-3,-3:no-sao=1:vbv-maxrate=20000:vbv-bufsize=80000:vbv-init=0.9

[microchip8]

Originally Posted by Hostile_18

I went with Opus in the end, everything can direct play no issues. My settings below are doing really well. Roughly about 2 and a half hours per disc. Someone on reddit thought I might be doing alot of complex maths and not using it with my settings, so was recommending faster presets? When I've looked into it compared to the medium preset the only setting I've got that adds a lot of time is "rect" If I turn it off one 60 second test video goes from 2.16 to encode to 1.30, with my slow preset settings below. Is Rect essential to my video quality or can I safely turn it off and save a lot of time? File size was roughly the same.

Star to Hex saved 8 seconds per 1 min test video, so not as dramatic, but again not to sure the visual cost.

libx265 -pix_fmt yuv420p10le -crf 17 -preset slow -x265-params aq-mode=1:no-open-gop=1:high-tier=1:level-idc=4.1:bframes=6:subme=4:rc-lookahead=120bratio=1.25:ctu=32:merange=26:deblock=-3,-3:no-sao=1:vbv-maxrate=20000:vbv-bufsize=80000:vbv-init=0.9

rect can be pretty beneficial. To speed it up, use limit-modes=1 which uses some magic to decide which partitions are unlikely to be selected as (best) candidates. If you use limit-modes=1 you might want to turn amp too (which depends on rect being on), although that one is not as beneficial as rect but virtually never hurts.

Might also want to turn off strong-intra-smoothing

[Hostile_18]

Originally Posted by microchip8

rect can be pretty beneficial. To speed it up, use limit-modes=1 which uses some magic to decide which partitions are unlikely to be selected as (best) candidates. If you use limit-modes=1 you might want to turn amp too (which depends on rect being on), although that one is not as beneficial as rect but virtually never hurts.

Might also want to turn off strong-intra-smoothing

I think Limit Mode=1 is on by default on Slow preset, at least according to this table; (I highlighted it to see what changed from the Slow Preset I was already on, and what didn't).



I could add "rskip=2" that saves a bit or time, but its been hard to distinguish if it has a noticeable effect on quality. It saves around as much time as turning rect=0, so that's a possibility.

I'll look into Strong Intra Smoothing, I did research here a while ago and there seemed to be some debate here if it actually made the image worse or not.

[microchip8]

I use rskip=2 with a threshold of 1 (which translates to 0.01, lowest possible = best quality) and it does speed up things. I have not noticed any issues with it and I've done around 100 encodes thus far with it.

[Z2697]

rskip=2 usually have more quality cost than rect=0.
strong-intra-smoothing is only effective on 32x32 intra blocks (max PU/TU size for intra), if the partitioning part of the default encoding process is not altered, the large blocks are usually smooth areas.
And, the smoothing only applies to reference pixels (reconstructed neighbors).

[microchip8]

I don't like anything with smoothing in its name or softening the image like SAO, and also find the deblocker at defaults to soften too much so I set it to -3:-3. Along with some other micro-optimizations I get a very detailed picture. I don't do 4k encodes as PC is too slow for that (around 2 fps speed) so scale down to Full HD using a Spline64 scaler from zimg. My Ryzen 9 5900X (stock max speed 3.7 GHz, overclocked by me to 4 GHz on all cores), I get between 6.5 and 8 fps speed, sometimes higher if source is very clean and compressible.

[GeoffreyA]

These are the settings I've settled on; though in terms of sharpness, it still loses to x264 if comparing frame for frame. No doubt, more could be squeezed out of it.

-preset slow -crf %crf% -x265-params level-idc=41:no-sao=1:deblock=-1,-1:aq-mode=1:cbqpoffs=-3:crqpoffs=-3:
                                     ctu=32:tu-intra-depth=3:tu-inter-depth=3:limit-tu=4:
                                     rd=4:rskip=2:rskip-edge-threshold=1:ref=4:limit-refs=3:subme=4:max-merge=4:
                                     no-open-gop=1:keyint=240:min-keyint=24:rc-lookahead=240:bframes=8:b-intra=0:weightb=1:fades=1:no-amp=1

[microchip8]

And these are mine

ref=4:me=umh:merange=52:subme=7:bframes=6:rd=4:rd-refine=0:qcomp=0.60:fades=1:strong-intra-smoothing=0:ctu=32:qg-size=16:sao=0:selective-sao=0:cu-lossless=0:cutree=1:tu-inter-depth=4:tu-intra-depth=4:max-merge=5:rskip=2:rskip-edge-threshold=1:rc-lookahead=80:lookahead-slices=0:aq-mode=1:aq-strength=1.1:rdoq-level=1:psy-rd=3.0:psy-rdoq=3.5:limit-modes=1:limit-refs=0:limit-tu=0:deblock=-3,-3:weightb=1:weightp=1:rect=1:amp=1:wpp=1:b-intra=1:b-adapt=2:b-pyramid=1:tskip=0:tskip-fast=0:fast-intra=0:early-skip=0:splitrd-skip=0:refine-mv=3:refine-intra=4:refine-inter=1:min-keyint=24:keyint=240

I've used in the past subme=5 but find subme=7 to subjectively provide a slightly more sharper image. It's not a sharpening filter in the classical sense but what it does is find more precisely the best possible estimation at subpixel level, thus providing more precise/better detail preservation which leads to the impression as if the image has been slightly sharpened like with a filter. It's a micro-optimization and since my CPU can take it, why not use it?

I also disable all lookahead-slices since I vaguely recall it has a negative impact on AQ?

[Hostile_18]

Thank you everyone, a lot to drive into and experiment with. I will report back my findings.

Has anyone found a good cpu between performance and power use? i.e the efficiency sweet spot. I am currently using a 9800x3d which is great, but it's attached to my gaming PC, so it's both my gaming pc and nas on 24/7. I may upgrade my i3 14gen server cpu if the time saving is worth it.

[Hostile_18]

One thing I have noticed is everyone's settings struggle with the OP screenshots from the first 10 minutes of Forest Gump Remaster. The grain on the wall in the corridor and the fine detail on the beige wallpaper at the table looks different on all our settings to the source.

I think that's a rare case though.

[Z2697]

If the retail price is considered, I'd go with previous gen, for pure power efficiency though, the latest gen is almost always the best.
You can even try ARM! It's efficiency has been better than x86(x86_64/x64) for years, and the performance side has lots of improvement in recent years, but I never used any, except -- of course -- my phones.

[Hostile_18]

Originally Posted by Z2697

If the retail price is considered, I'd go with previous gen, for pure power efficiency though, the latest gen is almost always the best.
You can even try ARM! It's efficiency has been better than x86(x86_64/x64) for years, and the performance side has lots of improvement in recent years, but I never used any, except -- of course -- my phones.

Tech Powerup has some relevant charts. Gets a bit complicated price wise when I factor in already having a 9800x3d but having to run it on a secondary system, or that I already have an 1700 motherboard socket in my NAS. Still the graphs are interesting.

[microchip8]

A few days ago, I read an article that the upcoming Intel Nova Lake desktop CPUs can draw up to 700 Watts!!! under full load! Jesus! Intel has totally lost it and has surpassed the power hungry Pentium 4 joke of many, many moons ago.

I would definitely NOT consider an Intel system for encoding. I'm on Linux, but their hybrid architecture still gives problems on Windows that the majority of people use.

What is the point of this hybrid E, LP and P cores? It's fine if the E/LP cores can significantly reduce power draw at idle, but once you start using the P cores for an extended period like during encoding, you might get a big surprise on your next power bill!

Yeah, AMD it is for the (near) future!

[microchip8]

Originally Posted by Hostile_18

One thing I have noticed is everyone's settings struggle with the OP screenshots from the first 10 minutes of Forest Gump Remaster. The grain on the wall in the corridor and the fine detail on the beige wallpaper at the table looks different on all our settings to the source.

I think that's a rare case though.

I'm not very sure, and this can go the other way, but x265 has a grain-optimized ratecontrol option --rc-grain - and also a --tune grain tuning. I have never used it myself but have read a few negative comments on here that it actually makes things worse. You might want to try it out on that problematic clip and see how it goes.

All in all, there's still a lot of improvements possible in x265, but it seems MultiCoreWare has (semi)-abandoned it

[Z2697]

rc-grain was designed for tune grain, or vice versa, whatever.
They aim for "homogeneous" of the QP in one frame so the grain won't look "blotchy".

[x264N00b]

Originally Posted by Hostile_18

One thing I have noticed is everyone's settings struggle with the OP screenshots from the first 10 minutes of Forest Gump Remaster. The grain on the wall in the corridor and the fine detail on the beige wallpaper at the table looks different on all our settings to the source.

I think that's a rare case though.

That's not very rare, it's a issue you have to deal with when it comes to high compression. It's about finding the right quality balance for the whole film where the bitrate for grainy content dosn't get bloated and flat and/or dark scenes with low contrast still remain with enough detail. So usually aq-mode 1 with CRF16 und preset slow delivers a overall very solid quality but sometimes it's just not enough. (bitrate)

What exactly is the bitrate in this encoded opening scene? You can analyze it with the tool ffbitrateviewer (second based view) or show it in MPC-HC. (view -> statistics -> current video bitrate)

I don't know how to solve the problem without overall decreasing the rate-factor even more or disabling cu-tree (usaually you chose a less low rate-factor then) or using x265 zones for this specific scene. Or try x264