Yes, you can write an application with a simple text editor. Alot of folks prefer it. They don't like the overhead of a huge development platform. As far as simple editors, PSPAD is nicer, as it highlights your syntax. After you have the source ready ( the .c or .cpp file), Then you compile it.
In simple terms, the compiler is a program that translates your c code into a language your computer can understand.
The program that you use to write c codes is not a compiler but editor. You can use any ASCI text editor to write codes and then compile them using your favorite c or c++ compiler. I reccomend using mingw from mingw.org.