I like the answer given by user6726, but want to address some other issues.
Generally, any kind of /e/ will be "halfway" between /a/ and /i/ as you divide the "vowel space" into three. To distinguish [ɛ] and [e], you generally have to divide the same space into four, so that [ɛ] will be noticeably closer in sound to /a/ and [e] will be noticeably closer to /i/. As user6726 mentioned, the specifics will vary from language to language and even dialect to dialect, generally depending on what kind or kinds of /a/ and /i/ exist in the language and what other sounds exist as well.
Here is my experience. I am more or less a native speaker of General American English and have studied a number of other languages and dialects to varying degrees of fluency.
First, I find there is both a physical and psychological element to distinguishing phonemes like these. In my own case, I have no trouble distinguishing them in languages that clearly require the distinctions, but in languages that either don't clearly make the distinction in all cases (e.g., French) or in which they don't make the distinction at all (e.g., Spanish), I also have trouble. In other words, in such cases I am unsure what vowel system "standard" speakers use or what vowel system I use even if I can achieve a "good" accent overall. I think this is because I hear so much variation among speakers that exhibit the same variation.
If you are specifically interested in the vowels of English, you should appreciate that its vowel system is pretty complex and has extreme variation between dialects. If you go to this link on Wikipedia and scroll down a little bit, you will see three charts showing the IPA vowels of the Received Pronunciation (i.e., the "standard" pronunciation of England), General American, and General Australian. Of the approximately twenty vowel and diphthong phonemes listed, there is not a single vowel where the phonetic description and symbol line up exactly across the three dialects. In other words, speakers of these dialects don't pronounce a single vowel the same way in the same words across the three dialects.
More specifically to your question, you will see in the charts that where one English dialect has [ɛ], one or both of the others often has [e] for the same phoneme. That is why no English speaker is going to care or maybe even particularly notice whether you pronounce "dress" as [drɛs] or [dres], besides the accent it might give you. There is also significant variation within American and English accents, so that speakers are generally used to hearing a great deal of variation in the pronunciation of vowels, even though most individuals will pronounce them consistently in only one way according to where they grew up.
You should also notice that among the three dialects listed, none of them has a simple opposition between [ɛ] and [e], except General Australian in the long varieties: [ɛ:] and [e:]. In General Australian, but not in the other two dialects, the only thing that distinguishes "bared" and "haired" from "bird" and "heared" is apparently that the former has [e:] and the latter has [ɛ:]. In my General American accent, I use lax/short [ɛ] with "r" coloring in "bared" and "haired." For "bird" and "heard," I use [ɚ], which is a symbol that does not appear in the charts, but only in footnote C and later in the text on the same Wikipedia page.
When I try to learn the vowels of a language, I try to read the most specific phonetic descriptions I can find; however, I then leave this analysis behind and try to imitate, not the sounds, but the overall accent. I find the two approaches work best for me and might work for you.