Communication is the key to any healthy relationship, and when I say any, I mean human and non-human. Previously, one of the biggest culprits for not understanding what you meant was Siri, but now it looks like it has some competition with the new Kinect system, which can control practically everything on your Xbox One—if it can understand you.
Before you start screaming in frustration at your brand new Xbox One, try recalibrating your Kinect Sensor.
Turn up the volume on your television to normal TV watching levels. Now on your controller, hit the Menu button located just left of the X, Y, A, B buttons.
On your Xbox One, select Settings, then Kinect. Now move to Kinect doesn't hear me. On the next menu (Let's check your audio) select Start audio check.
At this point, others have suggested turning the volume up as loud as it can go during the mic calibration on the Kinect (until it says too loud). Then lower it until you get just beneath the max threshold, then make sure that the room is a quiet as possible.
The sensor checks the background noise in the area and if there is not too much background noise, the system will automatically go to the next step. If there is too much background noise, you will need to minimize it as best you can and then run the audio check again.
During the audio check, the Kinect sensor will play a bunch of tones. If the tones are too low, a message appears asking you to turn up the volume of the speakers. This shouldn't occur if you do the threshold test listed above. Now, if the sound levels are loud enough, you'll see a message telling you the audio check was successful.
The system will take a few seconds to finalize calibration and then your should be good to command away.
For an extensive list of supported voice commands, as well as some more tips on using voice commands on your Xbox One, check out the Xbox One's support page.