Depending on the circumstances, it could be illegal. My best advice would be to consult with a lawyer. Usually a consultation is free to you.
Videotaping and Photography on Private Property
http://communications-media.lawyers.com/privacy-law/Videotaping-and-Photography-on-Private-Property.html
Sometimes, whether a photograph or videotape was legally taken depends on the location where it was taken, and the photographer's status to be on that property. Generally, the public has an implied right to be on the public portions of private property, such as in a restaurant or a stadium. However, that implied right does not extend to private areas of public accommodations, such as offices or the hallways of condominium buildings. It depends on the situation and the social expectation. If you think that you're getting away with something, then you probably are, at least until you're discovered.
Public Access
Sometimes, it may seem that the public has been given access to private property, especially when public use of the property is traditional. Examples of this are wilderness areas, hiking trails, swimming areas or access areas to a beach. If you make a mistake, you still may be in trouble, regardless of your belief that your access complied with the law. Don't depend on the lack of signs. In most places people are not required to post their land with no trespassing signs.
Things you should know about trespass:
•Extending a camera over a fence may be a trespass
•Flying over a property for the purpose of taking photos may be a trespass
•"No Trespassing" signs are not required
•If you have permission, don't abuse the permission
Exceptions to ordinary trespass laws:
•Taking photographs to document or lessen a disaster
•Taking photographs to document a crime
•You may take photographs or shoot video from your own property, onto private property
•You have permission to be on the property for other purposes
What to Do
The factors above are great to consider, but the answer to the question of whether or not you were trespassing is fact driven. That simply means that all the facts relative to your legal status on the property may be considered if the property owner files a complaint against you. The key is to use your common sense and err on the side of not trespassing.
An interesting case involves a private road, and a photograph shot of the private road sign, while standing on the private road. The property owner is seeking money damages for an invasion of privacy. It's all over the internet now and the public can see the private road sign for themselves. This area of law is ever-changing because of the availability of cheap and easy-to-use digital cameras. In this modern era, nearly everybody is a photographer or videographer, and the issue could affect you.
Nuisance
There is something in the law called a nuisance. In terms of photography and videotape, a nuisance is some activity that interferes with a property owner's reasonable use of his or her own property. It is difficult for a property owner to prevail in a nuisance action against a photographer or videographer because it is difficult to meet the legal burden needed to prove a claim. The annoyance to the property owner needs to be very real, and not just the result of the property owner's hypersensitivity or subjectivity. Take wildlife photos on your own land, even if the neighbors are animal rights activists, but don't take photos of nude models in your front yard.