Our users were getting annoyed by seeing the modifiedon field in Timeline activities header instead of the actual Activity timestamp. I started investigation with the background idea of this control got very less flexibility & I convinced the users like that in many community forum threads.

It's partially true about the Note records. Its going to be always modifiedon value & the cascading relationship behavior rule decides the date value based on Assign operation changing the owner field values.

Now the problem is actually Activity records, especially Phonecall entries for us.

I know there are few extra options for UCI (Unified interface) compared to classic web and Activities got extra configurations as well. On digging further we can create a custom card form & we can leverage it to show in Timeline control. That really helped here.

Lets start with the steps to add Timeline in any form.

Then open by double-clicking it. Make sure you are in right place like below - you will see the "Additional options".

Display activity header using has two values - namely Default format and Field labels
Display activities using has two values - namely Default fields and Card form (You can have multiple card forms to pick one)

We can change the behavior & layout using a custom card form. That's what I end-up with.
I played around a bit & used this particular combination of "Default format" with a new custom "Card form" to achieve the desired layout/behavior.

The default card form uses Priority & Modified on by default. Instead of messing with that - I used a new card form. The header can have 2 fields max - I put a party field & scheduled end time there. Then 3 fields in body - I used subject, party & description. 

Instead of default format - field labels may be used, and instead of card form - default fields can be used. But the combination of "Default format" + "Card form" works best.
Enjoy! Have fun with research & learning!!