Mild spoiler alert. It's impossible to talk about a story without them. Footnotes are a bit more spoilery but I figure you'll probably still forget most of it before getting to the story.
Maybe I should try writing a generic review on this one instead of a focused one since that would actually express how much I liked it (though if that's what you want, I think it's worth reading just for the cute girls[0]), but I always prefer reading the more focused reviews so I do want to try writing them even if I'm bad at it.

One main topic of this review is one specific object that takes a major role in the latter half of the story: the Yokodera-kun notes. These are (heavily implied to be) the exact first 11[1] volumes of Hentai Ouji to Warawanai Neko. written (over time) in-story as a way for one of the characters (main girl), Tsutsukakushi Tsukiko, as a record for the main character (Yokodera Youto) who is cursed[2] to never be able to keep any memories (or rather, I think they get given to Tsukiko). This makes for one part of their relationship - Tsukiko uses the record keeping as a way to feel like Youto needs her help, and so she strives to keep a detailed record of everything that happened. (The fact that it happens to be in easily digestible light novel form is a bonus!)
It's partly played for laughs, but I think it's pretty nice having the story include itself. There's other times where the narration implies the story telling itself in a certain way, but I haven't quite seen it like this. The main time they're mentioned is that, since Youto can't actually remember anything, he needs to read the notes in order to catch up and see what happened before. This means that he's only going in with (almost) the same level of knowledge that a reader would have at that moment, and so you can be rest assured that no details outside the scope of those notes will ever be mentioned, as they are essentially forgotten and wouldn't exist anymore. This is true in a normal story too as it feels bad to bring new concepts in super late into the game, but it being fully justified why the slice of life excerpts we get shown are the only ones that are ever relevent in future events are nice. (It's also excusable by a story being in past tense and so being told with full hindsight of what's to come ahead, but I like the idea that that's just not what happens.)
Near the beginning (as in volume 5), Tsukiko treats her knowing Youto in the past as her one major method of connection to Youto over the other girls as Youto was hanging out at the the Tsutsukakushi home since 10 years ago (that's when he lost his memories) since she has confidence issues over how the others could be a better fit. This is probably the point where the notes began in the first place (even if not used for the above purposes until vol9), since this was when she realized she needed another point of connection, as she discovered a girl who he had also met very early and he doesn't have memories of it anyway. In the epilogue volumes, once they've changed the world into their desired happy ending and the previous world now feels like a dream[3], the notes are used to dramatic effect as Tsukiko treats the notes as the way the world[4] should be and strives to shape their friend group to be like this (only without the further help of god, this time), and the burning the notes after Youto's confrontation about this exact thing symbolizes her finally letting go of that past and just be willing to accept whatever the future has in store, since there's no reason to keep holding on to what you're familiar with. Even at the very start, Youto has no real understanding for what makes Tsukiko obsess over the past so much, since he knew this from the beginning.

This theme of a character holding onto the past is extremely common in just about any story, not just this one. Remembering that Youto doesn't have much care for the past, it makes one easy way for him to be a hero helping the girls just being to have them let go of the past and just live on with their new highschool friends.
The most notable of these stories is with Tsutsukakushi Tsukushi, where she has a situation where she doesn't want to go to Italy to live with their grandparents because of how her mother died and she wants to preserve the family home exactly as it was[5]. Even when their house is being destroyed via an unrelated act of god[6], they consider the mementos of mom that were left behind to be important. This one isn't actually part of Youto's actions since Tsukiko (who does not have these memories of mom) is plenty convincing on her own and if you can only protect a few things, it's much more natural to pick what'll be useful for the future. After realizing that she's not so lonely after all, she never makes a huge deal of it again - though of course, now she has a more important reason to stay in Japan.
The other one that I liked was with Maimaki Mai (yes, this is this story's naming sceme) who, due to having lost a friend in the past[7], kept pushing herself away from making any close connections since she didn't want to hurt them and have them go away like the one she used to have. This one got solved after talking to Youto about these regrets, Youto responding that he's jealous of people who have memories at all, and having a brief (several day) body swap phenemonon for the two to learn the values in each others' lives. This problem gets solved by having Mai realize that she actually has close friends of her own and she can tell they wouldn't leave her, since Youto's relationships with his friends are essentially the same that she has with hers.
While there's others, this review reached my length goal and I don't think the others add as much to the point I'm trying to make, which is that this story's main point is that you should always live for the present instead of the past. Writing a diary (or notes) of things that happened in the past might be nice to look back on, but should never be treated as important as what'll happen next.

[0] And new fetishes to awaken to. There's a decent amount of stuff in here I've seen before and never really got, but having it show up with characters that I already like helps a fair bit... though there's nothing truly niche, of course; an ecchi scene won't sell well if it's something most people wouldn't get the point of.
[1] vol13 includes a scene of Tsukiko writing more story, which is probably supposed to be those last two volumes but I'm less sure on that. And they aren't the Yokodera-kun notes anymore by that point, either.
[2] Cursed, as in he told god to give his memories away to other people (or specifically "whoever needs it most", as is the usual job of this particular god). And then he never bothered getting his memories back, even though in volume 1 they figured out the mechanics of how it works and it was completely feasible to. (Though I think that's more because he decided he really doesn't need them as he just lives in the present.)
[3] Or, rather, no one remembers the previous world at all, making it more like Tsukiko's storybook world. Although it still happened, which is what does make it like a dream. (Though, their new world line just wasn't far enough away, so the world goes to be similar to how it was before, only just a little bit more peaceful.)
[4] You never really see any mentions of a part of the world apart from what's immediately relevant to the main group (after all, those aren't really related to Yokodera-kun and so don't make sense to be in the notes), so by "world" I really just mean the main group and other named characters. But for all I know, some rando in Italy (like the people Emi interacted with during her time there) could be living an entirely different life because of one of those changes.
[5] I would hope this is a very familiar plot point to you, since aside from minor details it shows up everywhere. Although in this case, it was actually Youto's memories that got given to her since she wanted more memories of the parents she wasn't able to live with.
[6] Tsukiko's wish to spend their last days in Japan with Youto, if I remember correctly. (How does that destroy the house? Read the book yourself to find out.)
[7] As it turns out, this happened because she moved and didn't realize that you can't receive letters unless the sender knows your address. I'm glad they come up with reasonable excuses for things like this. (By the way, in their new and improved world this is one of the things that got solved, except then the friend they lost was Youto instead since he joined them as a childhood friend to fix this in the first place.)