That "Map" is not outlining Sabine NWR, its outlining the corporate limits of Hackberry. Interesting that it includes a good portion of Sabine. The map in the hunting and fishing regulations shows the boundaries of Sabine NWR:
http://www.fws.gov/swlarefugecomplex...ochure2014.pdf
It is difficult to tell, but it is likely that you were partially within the NWR during your initial trip in, and once you turned around. However, you were not doing anything illegal. There are only certain areas that boat access is not allowed:
http://www.fws.gov/southeast/pubs/SWLAcomplex_Fish.pdf