SB 5.6.3

tathā coktam —
na kuryāt karhicit sakhyaṁ
 manasi hy anavasthite
yad-viśrambhāc cirāc cīrṇaṁ
 caskanda tapa aiśvaram
Synonyms: 
tathā — so; ca — and; uktam — it is said; na — never; kuryāt — should do; karhicit — at any time or with anyone; sakhyam — friendship; manasi — in the mind; hi — certainly; anavasthite — which is very restless; yat — in which; viśrambhāt — from placing too much faith; cirāt — for a long time; cīrṇam — practiced; caskanda — became disturbed; tapaḥ — the austerity; aiśvaram — of great personalities like Lord Śiva and the great sage Saubhari.
Translation: 
All the learned scholars have given their opinion. The mind is by nature very restless, and one should not make friends with it. If we place full confidence in the mind, it may cheat us at any moment. Even Lord Śiva became agitated upon seeing the Mohinī form of Lord Kṛṣṇa, and Saubhari Muni also fell down from the mature stage of yogic perfection.
Purport: 

The first business of one trying to advance in spiritual life is to control the mind and senses. As Śrī Kṛṣṇa says in Bhagavad-gītā (15.7):

mamaivāṁśo jīva-loke
 jīva-bhūtaḥ sanātanaḥ
manaḥ ṣaṣṭhānīndriyāṇi
 prakṛti-sthāni karṣati

Although the living entities are part and parcel of the Supreme Lord and are therefore in a transcendental position, they are still suffering in this material world and struggling for existence due to the mind and the senses. To get out of this false struggle for existence and become happy in the material world, one has to control the mind and senses and be detached from material conditions. One should never neglect austerities and penances; one should always perform them. Lord Ṛṣabhadeva personally showed us how to do this. In the Śrīmad-Bhāgavatam (9.19.17) it is specifically stated:

mātrā svasrā duhitrā vā
 nāviviktāsano bhavet
balavān indriya-grāmo
 vidvāṁsam api karṣati

A gṛhastha, vānaprastha, sannyāsī and brahmacārī should be very careful when associating with women. One is forbidden to sit down in a solitary place even with one’s mother, sister or daughter. In our Kṛṣṇa consciousness movement it has been very difficult to disassociate ourselves from women in our society, especially in Western countries. We are therefore sometimes criticized, but nonetheless we are trying to give everyone a chance to chant the Hare Kṛṣṇa mahā-mantra and thus advance spiritually. If we stick to the principle of chanting the Hare Kṛṣṇa mahā-mantra offenselessly, then, by the grace of Śrīla Haridāsa Ṭhākura, we may be saved from the allurement of women. However, if we are not very strict in chanting the Hare Kṛṣṇa mahā-mantra, we may at any time fall victim to women.