Ascertaining Ratzon HaShem: How do we know what G-d wants us to do?

Knowing G-d's Will  is a very deep and complicated topic which furthermore - while it has no clear answers - we are judged on how close we come to the right answer. I came across a very interesting and fundamental discussion in Minchas Asher of Rav Asher Weiss regarding last week's parsha. Below is some of the material from Rav Weiss (the first and last page). He deals with the issue of whether all our Torah obligations are included in halachos which are explicit in verses in derasha or whether we also have Torah obligations based on our inferences as to what G-d wants. 

You will note we are not dealing with Rabbinic ordinances of Chazal or contemporary gedolim. One example he discusses is the prohibition of tzar baalei chaim - unnecessary suffering of animals. It is agreed that this is in fact a Torah prohibition - but where is it stated? 

This is of course relevant to our discussion of the murder done at the Gay Parade. The murderer decided that it was Ratzon HaShem for him to randomly attack participants at the Parade. While it is clear that Shlissel is mentally ill - his decision making was based on his understanding of the Torah. Thus it is relevant to ask how applicable is his evaluation - to ignore the law of secular society not to kill as well as the Torah prohibition not to kill -  to the rest of us. To what degree can we assume that a mentally healthy adult will deviate severely from society's norms for the sake of higher principles i.e., the Will of G-d? What if any safeguards are there for an individual who is trying to ascertain what G-d's Will - not to harm others or themselves? Is there any restriction of determining G-d's Will to gedolim - or is it an activity that every Jew is required to participate in?

Let me make this a bit stronger. A number of years ago I participated in a series of meetings between religious and non-observant Jewish psychotherapists for the purpose of learning how to bridge the gap between religious and non-religious Jews. It was organized by an intelligent non-religious therapist whose grandfather - who he had known - had been the last religious member of his family. When we first met he was notably nervous. I asked him him what was making him nervous. He replied simply - "I am not sure the religious Jews won't kill me if they have a chance" He was referring to the religious members of this group - including me!

This fear of religious Jews - killing their non-religious brethen to fulfill G-d's Will - is not the limited to paranoid people and it isn't a rare phenomenon. Religious Jews can be scary people (I know charedim  who are uncomfortable going into Meah Shearim) - partly because they are beholden to a higher authority. The question is how are the requirements of that higher authority determined. Furthermore are there parameters such as that what we think is G-d's Will must conform to Darchei Noam i.e., be objectively pleasant? Not violate the law of the land? Not create a Chilul HaShem? Not harm others?