English Translation
To give him a fair evaluation, he is not lazy.
Very unnatural robotic translation… Here is my alternative translation:
実際のところ、彼は怠け者とはいえない。（じっさいのところ、かれはなまけものとはいえない。）
彼は怠惰だ is unnatural. We rarely use 怠惰だ as an adjective to directly describe a person. 怠け者 as a noun is much better in this context.
公平に評価すれば is an unnatural word-for-word translation in this context. The meaning in English is “taken into account all aspects/facts, he shouldn’t be labelled as a lazy person”.
実際のところ～とは言えない in Japanese is a set phrase when you say “in fact, describing Mr./Ms. X as Y is overstated”. 実際のところ (in fact) is equal to “taken into account all aspects/facts”. It’s obvious that it was evaluated based on facts (i.e. fair evaluation).
You can replace 実際のところ、彼は怠け者とはいえない with 実際のところ、彼は怠け者ではない, but the latter sounds like a definitive statement. So, I prefer inserting とはいえない (lit. = cannot be said that…) in order to make the tone moderate – an evaluation is often subjective, so a strong denial such as ではない is not recommended.