When you know God has given you a direction to lead your group into and through, it is common for people to question you. Just expect it! They question Moses and Aaron in Exodus 5 (and many times after) when their vision from God initially brought the people much harder work and oppression. Once they came to Moses and Aaron to complain, the first reaction Moses makes is to bring it back to God and ask why He brought harm to the people. Moses knew very clearly what God had asked him to do, and now the very people he was there to deliver are angry with him for handling it the way he did. He does not give up on the vision or run away from it. He simply comes back to God as if to say, "YOU told me to do this, and now look what is happening!"
He was not giving up, but he was bringing God's vision and plan back to him and asking for answers.