Input lag is the delay between pressing a button and seeing the game react. The potential causes for "input lag" are described below (steps which have negligible contributions to the input lag have been omitted). Each step in the process increases "input lag", however the net result may be unnoticeable if the overall "input lag" is low enough.
This is the lag caused by the digital televisions and monitors. Image processing (such as upscaling, 100 Hz, motion smoothing, edge smoothing) takes time and therefore adds some degree of input lag. It is generally considered that input lag of a television below 30ms is not noticeable. Discussions on gaming forums tend to agree with this value. Once the frame has been processed, the final step is the pixel response time for the pixel to display the correct color for the new frame.
CRT TVs and monitors have no display lag.
If you're using Windows Vista/7, having Aero enabled will add a noticeable amount of input lag because it forces vertical synchronization at the OS-level.
GPU driver latency
There is video latency caused by the GL drivers in Windows/Linux. Both the GLX X11 and Windows GL/D3D drivers are are full of hacks, codepaths, and buffer schemes that cater to benchmarking applications and games. This is counter productive when the aim is low-latency audio and video synchronization for emulators. You don't want all this stuff going on in the background.
Hard GPU sync options in some emulator frontends can reduce or remove latency from buffering at the possible expense of performance.
This can be avoided by using KMS and DRM/EGL, specifically on Linux. By using these modes, the user is in control of front and back buffers and don't have to rely on APIs, so that they can find where and when a frame was dropped and how to act accordingly with that in mind. It is advisable to get the latest driver to improve performance, as notable graphics chip manufacturers (e.g. AMD, Nvidia) do not find KMS a priority. Intel graphics chips, however, should be fine regardless, but it is still advisable to update drivers.
Low-level APIs such as Vulkan give the user control over buffering, and may lower latency without resource-heavy solutions like hard GPU sync. However, there is evidence that OpenGL has lower latency than Vulkan in some instances.
For wired controllers, this lag is negligible. For wireless controllers, opinions vary as to the effect of this lag. Some noticed extra lag when using a wireless controller, while others didn't.
Typical overall response times
Testing has found that overall "input lag" (from controller input to display response) times of approximately 200ms are distracting to the user. It also appears that (excluding the monitor/television display lag) 133ms is an average response time and the most sensitive games achieve response times of 67ms (again, excluding display lag).
Ways to reduce input lag
To disable Windows Aero under Windows Vista/7, select the Basic or Classic theme under Control Center > Personalization, or disable desktop composition under .exe properties > Compatibility. Some emulators and frontends allow you to disable desktop composition without having to switch themes. Desktop composition will also be disabled by playing under non-windowed full screen. In Windows 8 and later, desktop composition cannot be disabled manually.
Some emulator frontends like RetroArch or GroovyMAME have the option to delay processing of emulation for a few milliseconds until right before a vsync occurs, which causes inputs to be polled immediately before your display refreshes instead at the beginning of the 16.7ms (for 60 fps) vsync period. The amount of time you can use frame delay without dropping frames is dependent on the performance of the emulator on your machine.
If you don't have a CRT or can't be bothered with one, you can mitigate input lag on LCDs by setting the display to game mode if available. This turns off some post-processing effects that introduce lag.