Core MIDI and Friends
Published on Nov 7, 2011
CocoaHeads Ann Arbor presentation on handling MIDI events in Mac OS X and iOS
-
Ben SmileyI've written a series of compreshensive CoreMidi tutorials at: http://www.deluge.co/?q=all-things-midi4 years ago
Core MIDI and Friends
- 1. CoreMIDI and FriendsChris Adamson, CocoaHeads Ann Arbor Sept. 8, 2011
- 2. Road Map• MIDI basics• MIDI APIs on OSX• MIDI APIs on iOS
- 3. MIDI Basics• MIDI = Musical Instrument Digital Interface • Circa 1982!• Sends messages between devices, not sounds • Notes to play (pitch, impact, vibrato), patch selection, timing synchronization, etc.
- 4. MIDI Terms• Device — participant in a MIDI network• Endpoint — one connection into or out of a device• Source — endpoint that sends data• Destination — endpoint that receives data
- 5. MIDI MessagesSTATUS DATA 1 DATA 2
- 6. MIDI Messages• Channel Voice Messages — Note On, Note Off, After-touch, Pitch wheel • High nybble of status is command, low nybble is channel number• Channel Mode, System Messages• Various extensions to the spec over the yearshttp://www.midi.org/techspecs/midimessages.php
- 7. MIDI APIs on OSX• CoreMIDI — Implementation of MIDI messaging• Instrument Units — Audio Units that generate sound in response to MIDI events• Music Device — Sends commands to Instrument Units• Music Sequence / Music Player — Plays MIDI (.mid) files
- 8. Core MIDI• C-based API in Audio Toolbox • Largely similar to other Core Audio APIs • Uses Core Foundation opaque types, memory management, etc.
- 9. Core MIDI Types• MIDIClientRef — Holds state for your MIDI session• MIDIDeviceRef — A device (real or virtual) with “entities” (MIDIEntityRef), which are logical sub-systems• MIDIEndPointRef — Source or destination• MIDIPortRef — An input or output port
- 10. Creating client & input port!MIDIClientRef client = NULL;!MIDIClientCreate(CFSTR("Core MIDI to System Sounds Demo"), MyMIDINotifyProc, self, &client);!!MIDIPortRef inPort = NULL;!MIDIInputPortCreate(client, CFSTR("Input port"), MyMIDIReadProc, self, &inPort);
- 11. Discovering and connecting sources!unsigned long sourceCount = MIDIGetNumberOfSources();!for (int i = 0; i < sourceCount; ++i) {!! MIDIEndpointRef src = MIDIGetSource(i);!! CFStringRef endpointName = NULL;!! OSStatus nameErr = MIDIObjectGetStringProperty(src, kMIDIPropertyName, &endpointName);!! if (noErr == nameErr) {!! ! NSLog (@" source %d: %@n", i, endpointName);!! }!! MIDIPortConnectSource(inPort, src, NULL);!}
- 12. Read Proc• Callback function indicated by MIDIInputPortCreate() • Receives MIDIPacketList • numPackets and MIDIPacket[] • Receives user-info/context pointers that you set up in MIDIInputPortCreate() and MIDIPortConnectSource()
- 13. MIDIPacket• Contains timeStamp, length, and Byte[]• Byte[] data is the MIDI message • data[0]: status • data[1]: MIDI Data 1 • data[2] (if present): MIDI Data 2
- 14. Parsing a NOTE ON message!MIDIPacket *packet = (MIDIPacket *)pktlist->packet;!!Byte midiCommand = packet->data[0] >> 4;!// is it a note-on?!if (midiCommand == 0x09) {!! Byte note = packet->data[1] & 0x7F;!! Byte velocity = packet->data[2] & 0x7F;
- 15. Now What?
- 16. Instrument Units• Audio Units that generate sound in response to MIDI events • kAudioUnitType_MusicDevice• Do not currently exist on iOS
- 17. Instrument to audio H/W I/O Unit Unit AU Graph
- 18. Instrument to audio H/W Effect Unit I/O Unit Unit AU Graph
- 19. Instrument to audio H/W I/O Unit Unit AU Graph
- 20. Mu si cD ev ic eM ID IE ve nt () Instrument to audio H/W I/O Unit Unit AU Graph
- 21. MusicDevice.h• Small API to deliver MIDI events to instrument units• Not in Xcode documentation. Check out the header file • Only 4 functions • MusicDeviceMIDIEvent() sends status, data1, data2 to a MusicDeviceComponent (i.e., an instrument Audio Unit)
- 22. Demo
- 23. MIDI on iOS• Core MIDI added in iOS 4.2• Device connectivity is via dock port • Custom hardware • iPad Camera Connection Kit
- 24. MIDI via the CCK• MIDI-to-USB adapters semi-officially blessed by Apple • Adapter must be USB MIDI Class- compliant (i.e., doesn’t need drivers) • Bus-powered devices may not work, due to low power supplied by iPad. Powered USB devices generally work.• http://iosmidi.com/devices/
- 25. Playing sounds on iOS• No instrument units on iOS. Options: • Synthesize your own with a render callback • Play sampled sounds with Audio Queue, AV Player, System Sounds, etc.
- 26. Demo
- 27. Demo
- 28. AUSampler• New instrument audio unit in Lion… and… … • Takes a sampled waveform and pitch-shifts it to make it into an instrument • Call with MusicDeviceMIDIEvent(), just like other instrument units
- 29. AUSampler• Configuration is a huge hassle • Huge and tota!y undocumented hassle • Build an .aupreset with AU Lab utility, or load DLS bank or SoundFont 2 files, or provide your own files • See WWDC Session 411 (“Music in iOS and MacOSX”), then file documentation bugs against absence of sample code
- 30. In Summary• Musicians love MIDI. Devices are cheap and plentiful• OSX and iOS love media, MIDI included • By comparison, Android has crap MIDI support (no javax.sound.midi, just .mid file support in android.media.JetPlayer)• If you’re doing Mac or iOS media apps, you should consider supporting MIDI device I/O
- 31. Also, you should buy my bookhttp://www.mypearsonstore.com/bookstore/product.asp?isbn=9780321636843 http://my.safaribooksonline.com/9780321636973 http://www.informit.com/promotions/promotion.aspx?promo=137039
南来地,北往的,上班的,下岗的,走过路过不要错过!
======================个性签名=====================
之前认为Apple 的iOS 设计的要比 Android 稳定,我错了吗?
下载的许多客户端程序/游戏程序,经常会Crash,是程序写的不好(内存泄漏?刚启动也会吗?)还是iOS本身的不稳定!!!
如果在Android手机中可以简单联接到ddms,就可以查看系统log,很容易看到程序为什么出错,在iPhone中如何得知呢?试试Organizer吧,分析一下Device logs,也许有用.
对于博客园里的网友,不敢称为叫"程序员"的人,你们攻击性太强,看来你们是不会想到我的用意的.园子里有不少人都非常喜欢Jeffrey,是因为它的第一版 框架设计 CLR via C#.
可是从第一版到现在的第三版,没有看到真正底层的东西,内容仅仅是比MSDN文档更丰富一些,可能是我的要求太高了吧.
也就是因为它很多时候会接触到微软开发人员,会经常聊聊某些问题而已,而它又将这些问题反应到书中.也许它就像一个小记者.
它的年龄大我们不多,我的孩子与它小儿子一般大,如果我能向它那样出入微软与它们开发人员长时间交流,不仅仅会牛成它这样.....
可是微软的开发人员不会扔太多时间在它这儿的.所以它会整天追着这个,赶它那个..屁颠个不停吧...
而它的另一版被称为好书的 Windows核心编程,更是没有什么深度可言,仅仅是将windows提供的api,以及内核功能再重申了一遍.
这些书对晋及编程知识是有些贡献的,再说一遍我不是在匾低谁,说说想法而已.