A PKG file, diminutive of "Package", is simply a package of installation files. These files are used by the Mac Operating System and are executed when a user wants to install a program or software on a computer.
To view PKG file contents you can rename it to ZIP and open it using any unarchiver. Every PKG file is compressed using ZIP algorithm and can be extracted. But extraction is not the same as installation process, since installation can contain multiple instructions on how to modify operating system.
Yes, PKG files are compressed binary files. Though files located inside PKG arhive could be textual and contain configuration files, instructions how to install PKG file contents.