Are you seeing posts in your timeline from accounts you aren't following?
Don't worry, it's not an algorithm! You're probably just seeing posts which contain hashtags that you are following.
To check which hashtags you are following on Mastodon:
1. Go to your profile page
2. Depending on what app or interface you are using, click # at the top or click ... and then "Followed hashtags" (some apps may have other ways of doing this)
You can unfollow any tags you don't want to see.