An e-mail message’s confidentiality is protected when encrypted with the receiver’s public key, because he is the only one able to decrypt the message. The sender is not supposed to have the receiver’s private key. By encrypting a message with its private key, anybody possessing the corresponding public key would be able to read the message. By encrypting the message with its public key, not even the receiver would be able to read the message.
Source: HARRIS, Shon, All-In-One CISSP Certification Exam Guide, McGraw-Hill/Osborne, 2002, chapter 8: Cryptography (page 517).