A GPU can already cull the polygons on a per-polygon basis, but this happens at rasterization/setup time which happens after vertex shading (using Vertex/Geometry/Hull/Domain shaders). This means that the GPU can still end up shading polygon vertices that will end up getting culled. If this happens, the GPU will have done a lot of work for nothing and is therefore a waste of GPU cycles.
For this reason, we try not waste GPU cycles by culling invisible geometry on the CPU side (or GPU side using DX10+ features such as DrawIndirect and/or predicated rendering) in batches of polygons by simply not calling draw for those batches.
Typically that batch of polygons can be represented by a sphere or box that surrounds all those polygons. This bounding sphere/box is then tested against the frustum and if it touches the frustum, the draw call is invoked for those polygons. If not, the draw call is skipped.
An octree is simply another way of determining which batches of geometry are visible, except it organizes that data in a hierarchical fashion. Octrees are not necessarily suitable for all geometry.
Some fairly good references that explain the sphere/frustum technique follow: