Assessing the Impact of Force Feedback in Musical Knobs on Performance and User Experience

Abstract

This paper examined how rotary force feedback in knobs can enhance control over musical techniques, focusing on both performance and user experience. To support our study, we developed the Bend-aid system, a web-based sequencer with pre-designed haptic modes for pitch modulation, integrated with TorqueTuner, a rotary haptic device that controls pitch through programmable haptic effects. Then, twenty musically trained participants evaluated three haptic modes (No-force feedback (No-FF), Spring, and Detent) by performing a vibrato mimicry task, rating their experience on a Likert scale, and providing qualitative feedback in post-experiment interviews. The study assessed objective performance metrics (Pitch Error and Pitch Deviation) and subjective user experience ratings (Comfort, Ease of Control, and Helpfulness) of each haptic mode. User experience results showed that participants found force feedback helpful. Performance results showed that the Detent mode significantly improved pitch accuracy and vibrato stability compared to No-FF, while the Spring mode did not show a similar improvement. Post-experiment interviews showed that preferences for Spring and Detent modes varied, and the applicants provided suggestions for future knob designs. These findings suggest that force feedback may enhance both control and the experience of control in rotary knobs, with potential applications for more nuanced control in DMIs.

1. Introduction

Musicians often rely on physical feedback to refine their techniques and achieve the desired sound and expressiveness [1] because haptic feedback is a key channel to assess the accuracy of their movements and the music instrument’s response [2]. However, it remains unexplored how force feedback can facilitate nuanced control in music creation while addressing key challenges such as usability [3]. In musical devices like synthesizers, knobs are ubiquitous and play a crucial role in determining the quality of the music. For example, in live performances, fine-grained knob control can make or break the performance, and the absence of distinct physical feedback on knobs limits the precision needed to control various sound parameters [4]. Our goal is to explore how adding rotary force feedback to these knobs can enhance control and improve user experience in the management of nuanced musical techniques.

To assess the impact of force feedback on knobs for accurately controlling musical parameters, we chose pitch modulation, specifically vibrato, as a case study. Pitch is a perceptual attribute of sound that allows humans to categorize tones on a scale from low to high, and it is primarily determined by the frequency of the sound wave [5]. We developed a Bend-aid interface, a web-based tool for Musical Instrument Digital Interface (MIDI) editing and vibrato synthesis (Available online: https://github.com/piaoziyue/Bend-aid-Actuators-2024, accessed on 11 November 2024). Bend-aid integrates with TorqueTuner [6,7] to provide programmable rotary force feedback, allowing musicians to apply haptic feedback in vibrato techniques (as Figure 1 shows). Then, we conducted a user study to evaluate the control of vibratos using TorqueTuner and Bend-aid with three haptic modes. The results were analyzed from two perspectives: (1) technical performance, measured by Pitch Error and Pitch Deviation from recorded pitch contours; (2) user experience, assessed through subjective Likert-scale ratings on Comfort, Ease of Control, and Helpfulness; and qualitative feedback obtained from post-experiment interviews. This study aims to provide insights into how integrating rotary force feedback into knobs can enhance both control and user experience in music-making, offering design recommendations for future knob development to support more nuanced performance and sound manipulation.

2. Related Work

2.1. Haptic Feedback for Enhancing Control in Music

Compared to acoustic instruments, some nuanced music techniques cannot be realized using DMIs due to an emphasis on sound generation flexibility and a lack of accurate control [8]. The initial exploration of haptics and force feedback in musical interfaces began at Association pour la Création et la Recherche sur les Outils d’Expression (ACROE) with the development of a piano-like force-feedback key [9]. Chu found that tasks using audio mapped to vibrotactile feedback were more effective than those without it in audio editing [10], though it remains unclear whether these improvements can be generalized across different tasks and goals [11].

Today, musical haptics is an emerging interdisciplinary field in broader musical scenarios [12]. Researchers are actively exploring the integration of haptic feedback into DMIs [12]. Many efforts to incorporate haptic feedback into virtual instruments focus on replicating the tactile sensations of acoustic instruments. Examples include virtual drums with vibrotactile actuators [13] and virtual pianos using mid-air haptic displays [14]. For string instruments, robotic arms and vibrotactile actuators have been used to simulate techniques like plucking, bowing, and rubbing [15,16]. Tools like the Haptic Wave, developed for visually impaired producers and engineers, further illustrate the potential of haptic feedback in enhancing tasks such as audio effects editing [17,18]. These advancements highlight the potential of haptic feedback to improve control in DMIs, emphasizing the importance of understanding user perception of force feedback and developing haptic effects that enhance usability as key areas of ongoing research [11].

2.2. Rotary Force-Feedback Controllers

Knobs are designed for accurate and fine motor control [18,19] and are ubiquitous in daily life, from door locks to kitchen appliances and vehicle control panels, as well as in professional devices like medical equipment [20]. Integrating rotary force feedback can further enhance their functionality by offering angular position feedback and either continuous or discrete force feedback [21,22].

Knobs play a crucial role in the manipulation of music, such as controlling audio via MIDI keyboards or adjusting audiovisual elements in live performances using mixers [23,24]. For instance, the precision and ease of use make knobs ideal for tasks like adjusting frequency ranges or controlling pitch in synthesizers [4]. However, knobs in DMIs have traditionally been overlooked for innovation [4], and the lack of haptic feedback can cause performers to overshoot parameters during adjustments. Rotary force-feedback knobs can enhance the functionality of knobs in musical interfaces [18]. Devices like THE PLANK and the D’groove system integrate haptic feedback for more accurate control [25,26]. While a device like TorqueTuner, which uses direct current (DC) motors to deliver programmable force feedback, has shown promise, more experimental validation is required for the effectiveness of force feedback knobs in DMI applications [6,7].

3. System Design

We developed a system that integrates TorqueTuner and Bend-aid for pitch modulation. TorqueTuner was initially designed as a self-contained haptic knob for use with an existing DMI, the T-stick, which allows performers to control sound through sensors that respond to physical gestures and pressure sensitivity [27]. TorqueTuner expanded the interaction of DMI performance by providing real-time rotary force feedback [6]. A standalone version of TorqueTuner [7] was later created to support more advanced haptic designs. For this study, we enabled bi-directional communication in the latest version of TorqueTuner by optimizing bi-directional communication and enhancing force feedback control. Additionally, we introduced a custom 3D-printed grip to improve the grip during the vibrato experiment (see Figure 2).

We developed of the Bend-aid interface (as Figure 3 shows), which allows users to synthesize vibrato by controlling pitch through the rotation of the TorqueTuner, offering customizable force feedback modes. The following section focuses on the detailed design of Bend-aid’s visual interface and haptic modes.

3.1. Visual Interface Design

3.1.1. MIDI Sequencer

We developed the MIDI sequencer, adapted from the powerPianoRoll interface [28] and implemented using the Tone.js framework and a string instrument sampler [29]. The sequencer replicates common MIDI editing interactions found in Digital Audio Workstations (DAWs), a type of software used for creating and editing music, ensuring intuitive note manipulation. It supports functions similar to popular DAWs such as Logic Pro and Ableton Live, including:

• Double-click on an empty cell to add a note.
• Click on an existing note to highlight a selection.
• Drag a note to move it.
• Drag the beginning and end of a note to resize it.

The MIDI sequencer supports the editing of long pieces via a horizontal slider below it and can handle a wide range of notes, from low C1 to high G8. Notes can be manually entered, imported via the “upload” button or cleared with the “clear” button.

3.1.2. TorqueTuner Visualizer

The TorqueTuner visualizer in Bend-aid enables users to monitor the knob’s status in real-time. It shows the rotation angle using a small blue circle that moves around a larger circular indicator, acting as a pointer. Additionally, a blue bar positioned below the angle pointer represents the current torque value. When switching haptic modes, the current knob position is set as the zero point, and the angle and torque are reset until the next interaction. The angle contour is plotted below the MIDI sequencer, aligned with the timing of the notes. We can save the pitch contour data as a CSV file by clicking the “save” button.

3.1.3. Data Communication

The schematic of the communication between TorqueTuner and Bend-aid is illustrated in Figure 4. Once connected via USB, the TorqueTuner’s ESP32 Board communicates with Bend-aid by sending real-time data, including torque, angle, and velocity, to the serial port. These data are processed by Bend-aid to modulate the pitch of MIDI notes and update other internal variables, which are then displayed on the Bend-aid interface (e.g., through the TorqueTuner Visualizer). In return, Bend-aid can also send control signals back to the TorqueTuner. For example, when a new haptic mode is selected within the Bend-aid interface, the data processing unit sends the updated mode and settings to the ESP32 Board in TorqueTuner, which then activates the haptic mode setting. This change is reflected on both the Bend-aid interface and the TorqueTuner’s LCD.

3.2. Haptic Modes Design

The haptic modes in Bend-aid (Figure 5) are designed to extend the feedback mechanisms found in traditional synthesizer knobs. From our experience with popular synthesizers, we identified two main types of haptic feedback: constant resistance and incremental clicks. Based on the two types, some knobs offered additional tactile cues, such as mechanical stops for limited rotation, increased resistance at midpoints, or continuous rotation without angle limits. While these modes work well for simple tasks like adjusting volume, they are less suitable for accurate, continuous adjustments.

After developing the Bend-aid interface, we focused on designing and identifying the best modes for vibrato modulation. During preliminary testing, we invited two digital musical instrument designers specializing in haptic applications to test our system. We used Bend-aid to explore various modes, including Free (constant passive residual torque) and Vibrato (sinusoidal torque changes), as well as Magnet, Friction, Inertia, and Spin, which were adapted from the original self-contained haptic modes of TorqueTuner [6]. For the vibrato task, which involved subtle pitch changes, participants struggled when the pitch range was mapped to a large knob angle. Therefore, we selected Spring and Detent, along with No-FF, as the initial candidates for this exploratory study, focusing on a 0–45 degree range of haptic feedback.

3.2.1. No-Force Feedback Mode

The No-force feedback (No-FF) mode produces no force feedback as the motor is not actuated during knob rotation and serves as the control condition in our experiments. This mode allows us to compare outcomes with and without haptic force feedback, with the same user interface, highlighting the impact of the other modes on the user’s ability to control vibrato. This mode acts as a baseline, allowing us to isolate the effects of haptic feedback without needing to alter the user interface, ensuring that any observed changes are attributed solely to the haptic feedback rather than other physical aspects of the interface.

3.2.2. Spring Mode

In the Spring mode, rotary force feedback gradually increases as the knob is turned, similar to the common resistance pattern in pitch bend wheels. Although interacting with a pitch bend wheel requires holding and moving it horizontally—unlike turning a knob—testing this force feedback mode on a knob could yield interesting results, as this type of feedback is commonly used in pitch modulation. When the knob is released after being turned, it returns to its original position due to the spring’s torque. However, inertia can cause it to overshoot slightly, momentarily rotating past the original point to the opposite side.

3.2.3. Detent Mode

To provide a tactile cue at a specific angle, we developed a mode inspired by the mechanical stops of traditional knobs. In the first phase, minimal force feedback allows smooth movement. In the second phase, the force feedback increases rapidly, similar to a linear spring with a sharper slope to signal an approaching stop. The third phase offers a gradual increase in feedback until it reaches zero, indicating the end point.

4. Methods

This study explored the impact of three feedback modes (No-FF, Spring, and Detent) on enhancing control in the vibrato technique in terms of technique performance and user experience perspectives. Participants were instructed to replicate a sample vibrato played on a string instrument with the same timbre as the Bend-aid sampler and then compare it to their own performance using the TorqueTuner interface. They also visually monitored the timing of the notes and the TorqueTuner’s status and created vibratos using the Bend-aid interface. To avoid order effects, the sequence of haptic modes was counterbalanced. Additionally, all haptic modes were paired with the same visual feedback provided by the Bend-aid interface, ensuring consistent visual conditions across all modes. After each mode, participants rated their experience on a Likert scale for Comfort, Ease of Control, and Helpfulness and provided qualitative feedback on preferences and challenges during post-experiment interviews.

4.1. Participants

A pilot study involving one musically trained and one untrained individual revealed that musical training significantly affected task performance, with untrained individuals struggling to differentiate pitch variations in the vibrato. Therefore, we recruited musically trained participants, defined as those with at least 6 years of experience in a musical instrument or singing and without diagnosed auditory, visual, and motor impairments. Through the McGill University Schulich School of Music’s music technology events, we recruited 20 musically trained participants (7 females, 13 males), with an average of 15.6 years of musical training, and their information demography was listed in

Table 1. Participant codes, gender, primary instrument with years of experience, and secondary instrument with years of experience. All participants have over 6 years of experience with their primary and secondary instruments. Secondary instruments with less than 6 years of experience, as reported in the pre-experiment questionnaire, are shown in italics, indicating they are not considered main instruments in Section 5.2.2, Post-Experiment Thematic Analysis.

	Gender	First Instrument, Years of Experience	Second Instrument, Years of Experience
P1	Female	Piano, 14 years	–
P2	Male	Piano, 15 years	Flute, 15 years
P3	Male	Harmonium, 20 years	Voice, 1 year
P4	Female	Piano, 21 years	Flute, 10 years
P5	Male	Piano, 6 years	Guqin, 1 year
P6	Male	Guitar, 6 years	Piano, 2 years
P7	Female	Piano, 10 years	–
P8	Female	Violin, 20 years	Viola, 7 years
P9	Male	Piano, 22 years	Voice, 21 years
P10	Male	Piano, 16 years	Guitar, 1 year
P11	Female	Piano, 18 years	Voice, 10 years
P12	Male	Violin, 24 years	Piano, 10 years
P13	Male	Tenor Trombone, 24 years	–
P14	Male	Piano, 10 years	Organ, 7 years
P15	Female	Percussion, 16 years	Saxophone, 3 years
P16	Male	Saxophone, 15 years	Piano, 12 years
P17	Male	Electric Bass, 25 years	Double Bass, 13 years
P18	Female	Guzheng, 10 years	Flute, 3 years
P19	Male	Guitar, 11 years	Piano, 5 years
P20	Male	Clarinet, 12 years	Piano, 12 years

. Participants were compensated in cash with USD 10 for their 30 min participation.

4.2. Procedure

The experiment began with an introduction to the study’s goals and procedures, followed by the collection of consent forms. Before the vibrato task, participants completed a pre-experiment questionnaire (

Table 2. Pre-experiment questionnaire.

Pre-Experiment Questionnaire	Questions
Q1	How many years of formal musical instrument training do you have (including singing)?
Q2	How many instruments can you play? Please list two of your most proficient instruments and the number of years you have played them.

) on their musical background. After a brief demonstration of vibrato control using the TorqueTuner with Bend-aid, participants familiarized themselves with the interface.

In the experiment, participants listened to an example audio clip lasting 14 s. The notes followed a 4/4 time signature at 120 beats per minute (bpm), with each note lasting six quarter notes (or 3 s). For the vibrato example (Figure 6), within each note, the vibrato oscillated three times around the base pitch of C3, reaching a peak amplitude of 3 half notes above C3 before returning to the base pitch. Participants were then instructed to replicate the sound using Bend-aid and the TorqueTuner. They were allowed multiple attempts to complete the task until they were satisfied with their performance. After successfully mimicking the vibrato in one haptic mode, participants saved the pitch contour and completed a Likert-scale questionnaire, rating their experience in terms of Comfort, Ease of Control, and Helpfulness for the task (

Table 3. Likert-scale questionnaire and post-experiment interview questions.

Likert-Scale Ratings	Explanations
Comfort	How physically stress-free did the haptic mode feel to you during the task? (1 = very uncomfortable, 10 = very comfortable)
Ease of control	How effortlessly can you adjust the angle from one position to another position? (1 = very difficult, 10 = very easy)
Helpfulness	How much does the haptic mode help you improve your performance in the vibrato task? (1 = not helpful, 10 = very helpful)
Post-Experiment Interview	Questions
Q1	Do you have any comments regarding the haptic modes? Please explain your reasoning.
Q2	Did you experience any issues using the interface to control vibrato?
Q3	Do you have any suggestions for improving interaction with the force feedback knob?

). They then proceeded through two additional sessions, each using a new haptic mode, and rated their experience using the same Likert scale. Finally, we conducted a semi-structured post-experiment interview to gather feedback on participant preferences, challenges with the haptic modes, and suggestions for improving the interface’s usability (Table 3).

4.3. Analysis

We analyzed the data from both technique performance and user experience perspectives. Since the Shapiro–Wilk and Kolmogorov–Smirnov tests indicated non-normal distributions for both technique performance and user experience Likert ratings, we applied non-parametric statistical methods, including the Friedman test and Wilcoxon signed-rank test, to assess both technique performance and Likert ratings.

4.3.1. Technique Performance

We selected two metrics to assess technique performance: Pitch Error and Deviation, which represent the accuracy and stability of the performance.

Pitch Error: This metric quantifies the accuracy of the vibrato technique by calculating the average absolute deviation of the detected pitch peaks from a predefined target pitch peak. In this task, the target pitch peak rises by three half steps from the original pitch. A lower Pitch Error value indicates greater accuracy in hitting the desired pitch peaks. The formula is:

$$\mathrm{Pitch}\mathrm{Error}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|{\displaystyle \frac{\mathrm{Filtered}{\mathrm{Peak}}_i-\mathrm{Target}\mathrm{Pitch}}{\mathrm{Target}\mathrm{Pitch}}}\right|$$

where n represents the number of valid pitch peaks detected.

Pitch Deviation: This measures the consistency of the vibrato by calculating the standard deviation of the detected pitch peaks. A lower standard deviation indicates more consistent performance, reflecting a stable vibrato technique. The formula is:

$$\mathrm{Pitch}\mathrm{Deviation}=\mathrm{Standard}\mathrm{Deviation}\mathrm{of}\mathrm{Filtered}\mathrm{Peaks}$$

4.3.2. User Experience

Likert Ratings: We focused on the usability and experimental dimensions adapted from the Haptic Experience evaluation model [30]. We chose the ratings of Comfort and Ease of Control, primarily focusing on experimental dimensions, while Helpfulness addresses the usability related to the actual task. The Helpfulness rating can be compared with the technique performance analysis examining the differences between participants’ perception and their actual task performance.

Interview Analysis: We did a thematic analysis based on the feedback that we obtained from the post-experiment interview. To facilitate the analysis of the interview, we used the participant codes, from P1 to P20, and mainly focused on their musical training experience based on their responses in the pre-experiment questionnaire.

5. Results

This section presents the findings of our study by analyzing participants’ vibrato technique performance and user experience.

5.1. Technique Performance

Six outliers for Pitch Error and one for Pitch Deviation were removed using the Interquartile Range (IQR) method, leaving a final dataset of 53 observations. Both metrics were normalized to a range of 0 to 1 for direct comparison by scaling the values based on the minimum and maximum values within each metric.

We employed the Friedman test to assess differences across the three haptic modes. As Figure 7 shows, the test indicated variation across haptic modes for both Pitch Error (${\chi }^2=7.60$, p = 0.022) and Pitch Deviation (${\chi }^2=6.40$, $p=0.041$). Post hoc Wilcoxon signed-rank tests were conducted to examine pairwise differences. For Pitch Error, a difference was observed between the Detent and No-FF modes ($W=13.0$, $p=0.007$), suggesting that Detent mode improved pitch accuracy compared to No-FF. No differences were detected between Spring and No-FF ($W=54.0$, $p=0.762$) or Spring and Detent ($W=27.0$, $p=0.064$), though there was a trend toward a difference between Spring and Detent. For Pitch Deviation, a difference was found between Detent and No-FF ($W=25.00$, $p=0.048$), indicating that Detent mode supported a more consistent vibrato compared to No-FF. Other comparisons showed no clear differences.

Overall, the analysis suggests that haptic modes influence both Pitch Error and Pitch Deviation in vibrato performance, with Detent mode generally better than No-FF in both metrics.

5.2. User Experience

5.2.1. Likert Ratings

In this section, we explored the impacts of three haptic modes on participants’ subjective ratings of their user experience, including Comfort, Ease of Control, and Helpfulness.

In Figure 8, we visualized the Likert-scale ratings for the three categories. We assessed the impact of the three haptic modes on participants’ subjective ratings of their user experience, focusing on three key metrics: Comfort, Ease of Control, and Helpfulness. No outliers were identified or removed using the IQR method. The ratings were then analyzed to assess differences across the haptic modes.

The Friedman test was used to examine within-subject differences in the three categories across the three haptic modes. The test revealed no significant differences across the haptic modes for Comfort (${\chi }^2=2.84$, $p=0.242$) or Ease of Control (${\chi }^2=2.03$, $p=0.363$), indicating that these aspects of user experience were not significantly influenced by the haptic modes. However, perceived Helpfulness varied across modes (${\chi }^2=6.77$, $p=0.034$).

Given the result for Helpfulness, post hoc Wilcoxon signed-rank tests were conducted to examine pairwise differences between haptic modes. Differences were observed between No-FF and Detent ($W=37.5$, $p=0.035$) and between No-FF and Spring ($W=18.0$, $p=0.016$), with No-FF perceived as less helpful than Detent and Spring. No difference was found between Detent and Spring ($W=74.0$, $p=0.610$). For Comfort and Ease of Control, while the Friedman test showed no overall influence of haptic modes, Spring was rated higher than No-FF for Ease of Control ($W=30.0$, $p=0.025$).

Overall, the analysis indicated that while haptic modes did not impact perceptions of Comfort or Ease of Control, they influenced the perceived Helpfulness, particularly in comparisons involving No-FF.

5.2.2. Post-Experiment Thematic Analysis

In this section, we conducted a thematic analysis of the post-experiment interviews and identified several key themes.

Preferences about Haptic Modes: In this subsection, we analyzed participant preferences while mentioning in parentheses the main musical instrument(s) that they play, as the type of musical instrument participants have expertise with may impact their preference for haptic modes.

Most participants favored one of the two force feedback modes, showing a clear preference for modes with force feedback over those without. P19 (Guitar), P13 (Tenor trombone), and P8 (Violin and viola) found the No-FF mode to be the least helpful for vibrato. P17 (Electric bass and double bass) noted that “Fast movements are difficult in No-FF mode”, and P5 (Piano) mentioned that “No-FF mode provides too little haptic cues”. However, a few participants preferred the No-FF mode for specific reasons. P11 (Piano and voice) said, “I prefer No-FF mode. When listening to the sample vibrato, my hands automatically start to imitate the motion, and the No-FF mode best translates my movement”. P17 (Electric bass and double bass) added, “With force feedback, the knob becomes hard to hold, limiting movement to small adjustments”. Similarly, P10 (Piano) found No-FF mode “more comfortable”, though they acknowledged that “modes with force feedback are more precise”.

Participants had varying preferences for Spring and Detent modes, each favored for different reasons. For Spring mode, P16 (Sax and piano) found it “useful for knowing position”, while P7 (Piano) mentioned it “helps accurately find the pitch point”. P8 (Violin and viola) and P4 (Piano and flute) noted that Spring mode is comfortable and helpful but takes time to get used to. P6 (Guitar) preferred Spring over Detent mode for its continuous feel. For Detent mode, P20 (Clarinet and piano) and P8 (Violin and viola) considered it the most helpful, as it feels similar to a string instrument. P9 (Piano and voice) preferred Detent mode, appreciating the tactile cue, while P5 (Piano) stated it is ideal for “fast, dense vibratos”, providing helpful force feedback.

Some participants disliked one of the force feedback modes because its force levels are too high. P3 (Harmonium) and P9 (Piano and voice) found Spring mode challenging due to the excessive force, with P9 stating, “A small amount of resistance is good, but Spring mode requires too much force”. Similarly, P7 (Piano) mentioned that Detent mode has “too much force” and is difficult to turn.

Flaws about Returning to Original Position: Several participants expected the knob to return to its original angle when released, as they would with a pitch bender on a keyboard. P5 (Piano) and P14 (Piano and Organ) noted that an automatic return-to-zero feature would have eased their control, as they found maintaining their position challenging without it. P11 (Piano and Voice) and P16 (Sax and Piano) shared that the absence of this feature affected their performance. P16 explained, “I didn’t turn the knob fully because I wanted to maintain vibrato speed and return to the original point quickly”, and P11 added, “If the first vibrato doesn’t return to zero, it can disrupt the following vibrato”. While some, like P4 (Piano and Flute), found the Detent mode easier for returning to the starting position, these preferences indicate the importance of an automatic return feature for specific user groups.

Suggestions about Future Interaction: For a more personalized knob, P15 mentioned, “People have different hand strengths, and sometimes I want to use both hands to turn the knob because the force level is too high for me”. P18 (Guzheng) suggested, “On my guzheng, vibrato differs across strings. Adding a mechanism to adjust haptic spring tension for different pitches could help with natural interaction”. Others highlighted potential broader uses for rotary force feedback. P13 (Tenor trombone) suggested, “Some synthesizer techniques may work better with rotary movements than vibrato”. P2 (Piano and flute) recognized the potential of haptic modes as an educational tool for exploring various instruments, particularly in understanding their timbre and techniques.

Some participants suggested alternative interactions to better facilitate vibratos. P15 (Percussion) noted, “With haptics, creating vibrato is easier, but a knob can be harder to control. A button or touchpad might be better”. P14 (Piano and organ) agreed, adding, “A button or pedal would make control easier”.

6. Discussion

6.1. Detent Haptic Mode Improves Accuracy, Spring Haptic Mode Aids Perceived Control

The Detent mode is generally superior to the No-FF mode in vibrato pitch accuracy and perceived Helpfulness. Participants showed lower Pitch Error and more stable vibrato using Detent mode, suggesting that it offered better control, particularly for tasks requiring fast, accurate pitch modulation. The tactile feedback in Detent mode enabled participants to intuitively adjust vibrato, making it ideal for the vibrato task and potentially applicable to applications that require real-time modulation and accurate adjustments.

While some participants found the Spring mode helpful, it did not significantly improve Pitch Error or Pitch Deviation. In fact, Pitch Error was slightly worse than in No-FF mode, with considerable variability. Although some people felt that Spring mode assisted in finding pitch positions, it took longer to adapt to. Participants rated Spring mode higher for Ease of Control, possibly due to feedback about its continuous and comfortable feel. The performance of the Spring mode showed promise but could be further improved with the addition of more haptic cues.

Jense and Eggen tested six types of self-designed knobs and found that participants desired multiple mechanisms, such as a spinning element controlling note sustenation and a blob shape controlling timbre [4]. Similarly, we believe that our Detent and Spring modes could serve different musical parameters or purposes and that additional haptic modes may prove useful in the future.

6.2. Future Study on Musical Background and Haptic Preferences

Initial observations showed signs of the relationship between musical background and haptic mode preferences, indicating a need for further investigation. Future designs could draw inspiration from acoustic tactile sensations to create more intuitive experiences. For example, applying wind instrument reed equations to haptic modes has shown promising results [31].

We hypothesize that designing haptic modes tailored to individuals with different musical backgrounds could better match their preferences and motor skills. Future research could investigate whether string instrument players rely more on haptic feedback and tend to dislike No-FF modes, while wind instrument players might favor continuous haptic modes like Spring due to their resemblance to mouth technique sensations. Additionally, musicians from various instrumental backgrounds may have higher expectations for haptic simulations, seeking more precise feedback. Understanding these nuances can help create adaptable haptic modes that cater to individual preferences, enhancing both user experience and musical expression.

6.3. Limitations and Future Force Feedback Knobs

Rotary force feedback is essential for future knob designs. For example, many participants needed the knob to automatically return to its original position, particularly during left turns. Force feedback can enable this by providing asymmetric forces for left and right turns. Designing force feedback tailored to specific tasks and user needs can improve usability. Even in mechanical knob design, tools like the TorqueTuner with Bend-aid can aid in rapid prototyping and testing of haptic modes, contributing to the industrial design process.

This study has some limitations. First, the absence of an automatic return-to-zero feature is a key limitation. Many participants identified this as a drawback, noting that this feature is essential for future pitch modulation applications, particularly in vibrato control. Additionally, neither the Detent nor Spring modes provided significantly more comfort than the No-FF mode. Future designs should prioritize enhancing comfort in both existing and new haptic modes to address this gap.

This study serves as a starting point, utilizing one-to-one mapping for pitch modulation. While our initial results are promising, future research could explore more advanced mappings, such as more finely-designed haptic modes and even force feedback linked to multiple audio variables like timbre and texture, as seen in knobs like in [32], enhancing musical expression. Tools for interactive haptic force feedback design, such as Feelix [33], could assist in refining these effects.

7. Conclusions

This study developed the Bend-Aid interface for TorqueTuner with different rotary force feedback modes for pitch modulation and explored the impact of different haptic modes on vibrato modulation from technique performance, user experience, and subjective feedback perspectives. Our findings suggest that incorporating force feedback into rotary knobs can enhance pitch accuracy of vibratos and user experience in music-related tasks. Detent mode, characterized by an initial light resistance that quickly increases as you approach a stop before tapering off, provided superior pitch accuracy and vibrato stability compared to the No-force feedback mode, making it more suitable for nuanced and continuous modulation tasks. In contrast, Spring mode, with its gradually increasing force feedback, was rated higher for Ease of Control, likely due to its smooth and continuous feel. However, it did not significantly improve performance metrics.

We provided recommendations for future haptic mode design and highlighted areas for further research, such as how musical backgrounds might influence haptic preferences. These insights can aid in developing rotary knobs for music hardware, such as synthesizers, and other haptic interfaces that enhance both technical performance and musical expression.