Within the framework there are multitudes of plugin packages that contain said elements and many of them are built on top of ffmpeg.
In both cases you are best off restricting things to what you actually use.
Different cases really I think both are good.
Gstreamer has a different model, chaining together plugins. Lots of overlap, but I think Gstreamer only has real traction because some silicon vendors use it.
ffmpeg's core functionality (encode, decode, streams, pipes, channels) are all implemented in `libav` which gstreamer links against.
ffmpeg and other media frameworks (Windows Media Foundation, Apple’s AVFramwork) only support static pipelines. You can use “switcher” components but the inputs are still static.
GStreamer is extremely special. The only thing that comes close was Microsoft’s DirectShow, which has since been replaced with Media Foundation which can’t do it. And while DirectShow did support it, it was fragile because many 3rd party filters did not support dynamic configuration.
GStreamer does use ffmpeg, but it just wraps the core encoder/decoder/filter code and discards the streams/graph/pipe part of ffmpeg.
FFmpeg doesn't do “pipelines”. It's a library, not a framework.
"GStreamer" doesn't link against libav. The GStreamer plugin "gst-libav" links against libav. If you're not using the gst-libav, you're not using ffmpeg. I'd bet a relatively small amount of GStreamer use cases use gst-libav; I typically see people use e.g x264dec and x264enc (from the x264 plugin) to decode and encode media, or, for hardware encoding/decoding, the v4l2enc and v4l2dec elements (or elements from a SoC vendor's plug-in such as gst-rockchip). It also has its own non-libav elements to handle container formats, pixel format conversion, scaling, etc, which are more natural to use since they're part of the core gstreamer plug-in packages rather than gst-libav.