Sé que era difícil.

English Translation

I know it was difficult.

“sé” here is a conjugation of “saber”, and not of “ser.” As such, it does not belong in the ser vs estar activity.