Call a lawyer. You really need one. There are all types of laws that pertain, including potentially a "prescriptive easement." You may want to send your neighbor written notice that he has no right to bulldoze this tunnel and you hope to settle the matter amicably.
With regard to California, to acquire a prescriptive easement requires that your use must be (a) in the open, not in secret, (b) notorious and obvious, clearly observable, (c) hostile, without the landowner's consent and (d) continuous, without interruption for the number of years required by state law. was there any written consent from the prior neighbor? If there was none, it could be argued that you qualify under (c) in the absence of a written agreement giving you the easement in writing.
Here is case law I found:
"To establish the elements of a prescriptive easement, the claimant must prove use of the property, for the statutory period of five years, which use has been (1) open and notorious; (2) continuous and uninterrupted; (3) hostile to the true owner; and (4) under claim of right." See Tadina v. Bowles, No. C032230 (Cal.App. Dist.3 10/25/2001). "These elements are designed to insure that the owner of the property being encroached has actual or constructive notice of the adverse use with sufficient time to take action." See Sunburst Estates II Homeowners Association v. Sunburst Estates I Homeowners Association, No. B143938 (Cal.App. Dist.2 11/29/2001)