[93a middle] Shmuel: Watching over a field from a booth (which sometimes even involves physical action of getting up to guard it) is not the same as actively working a field (which Biblically entitles one to food rights).
Sefer Hachinuch: If an activity is possible to be completed with no action, even when a physical action is involved, it is still considered a “non-action.” Hating someone for something that they did to us is an absolute Torah prohibition. The mindset of revenge is an Aveira, even without yet having done anything back to the other person. Therefore, if one does (Chas Vishalom) retaliate, this is also considered a ‘non-action’ [which carries no physical punishment by court (but is punishable by Heaven)]. Even when another person punches us in the face, we must know that it is all directly from Hashem (and we may not harbor any resentment). All good and seemingly bad is the precise WILL of G-d. [R’ Twersky: This is why Yosef was not mad at his brothers for what they did to him (since he knew they were just Hashem’s messengers)].
D-E-ep Thoughts: Whenever we get stuck in traffic, have a financial loss, or need to see a doctor, there is never any reason to get angry. Hashem is in complete control and everything that happens to us is a Gift from G-d. Aside from not being mad at a stick being yielded by its Master, we should always be full of joy since everything that happens to us is a blessing (even when disguised).