The opening and closing tags for php in an an html document are
Anything you see in between is php code. What you'll find in WordPress and most php driven web pages is php being turned on and off a lot. For example here are a few lines from the page.php file.
The first and last lines are ordinary html. The 3 in between are php, though you'll recognize some html inside it. In your first attempt you probably don't want to change the parts inside the php tags, but over time you'll be comfortable doing just that. With at first line though you can add an id or another class or change the whole thing from a div to a span and then change the closing tag at the end.
<?php the_content(); ?>
<?php wp_link_pages( array( 'before' => '<div class="page-link">' . __( 'Pages:', 'twentyten' ), 'after' => '</div>' ) ); ?>
<?php edit_post_link( __( 'Edit', 'twentyten' ), '<span class="edit-link">', '</span>' ); ?>
</div><!-- .entry-content -->
In the beginning don't worry too much about what everything is doing. Just know it works. As you feel more comfortable start searching the codex or general search engines for some of the template_tags you see. For example in the code above you might search for the_content(). The name itself gives you some clue about what it's doing. It displays the content of a blog post or page. Searching though will give you some clues about how to use it and how you might alter it's behavior a little. In the beginning though you can just trust that it displays the content of your page. You can though look at the html source that gets put out. For example the_content will display html tags along with the content. You can still style those tags with css. What will be more difficult is changing the html that is produced.
You can start from scratch. It's more that you'll find it easier in the beginning to modify an existing theme until you feel more comfortable with what all the template_tags are doing.
So it is more about editing one of the existing themes than starting from scratch?
The loop is a few lines of php template_tags that grab and display the content of posts and pages from the database. It's not a separate file. The code will be present inside several files. Generally any time you want to have post and page content displayed in a page the loop will be present.
I hope I answered all your questions. Feel free to ask more. The best way to learn this by the way is to have WordPress set up somewhere and play around with the theme files. Have a backup of the theme just in case and then started editing the files to see what happens. The more you work in the WordPress code the easier it will be to understand what's going on. You won't figure out everything over night, but you will figure out enough within the month to be able to work with themes.