Privacy & Foss advocate, and Linux user. Dyslexic
Ace 🖤🩶🤍💜

Anti Commercial-AI license (CC BY-NC-SA 4.0)

  • 13 Posts
  • 596 Comments
Joined 1 year ago
cake
Cake day: June 14th, 2023

help-circle













  • However, to process more sophisticated requests, Apple Intelligence needs to be able to enlist help from larger, more complex models in the cloud. For these cloud requests to live up to the security and privacy guarantees that our users expect from our devices, the traditional cloud service security model isn’t a viable starting point. Instead, we need to bring our industry-leading device security model, for the first time ever, to the cloud.

    As stated above, Private cloud compute has nothing to do with the OS level AI itself. ರ⁠_⁠ರ That’s in the cloud not on device.

    While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.

    As stated here, it still has the same issue of not being 100% verifiable, they only publish a few code snippets they deam “security-critical”, it doesn’t allow us to verify the handling of user data.

    • It’s difficult to provide runtime transparency for AI in the cloud.
      Cloud AI services are opaque: providers do not typically specify details of the software stack they are using to run their services, and those details are often considered proprietary. Even if a cloud AI service relied only on open source software, which is inspectable by security researchers, there is no widely deployed way for a user device (or browser) to confirm that the service it’s connecting to is running an unmodified version of the software that it purports to run, or to detect that the software running on the service has changed.

    Adding to what it says here, if the on device AI is compromised in anyway, be it from an attacker or Apple themselves then PCC is rendered irrelevant regardless if PCC were open source or not.

    Additionally, I’ll raise the issue that this entire blog is nothing but just that a blog, nothing stated here is legally binding, so any claims of how they handled user data is irrelevant and can easily be dismissed as marketing.




  • Their keynotes are irrelevant, their official privacy policies and legal disclosures take precedence over marketing claims or statements made in keynotes or presentations. Apple’s privacy policy states that the company collects data necessary to provide and improve its products and services. The OS-level AI would fall under this category, allowing Apple to collect data processed by the AI for improving its functionality and models. Apple’s keynotes and marketing materials do not carry legal weight when it comes to their data practices. With the AI system operating at the OS level, it likely has access to a wide range of user data, including text inputs, conversations, and potentially other sensitive information.


  • Apple claimed that their privacy could be independently audited and verified.

    How? The only way to truly be able to do that to a 100% verifiable degree is if it were open source, and I highly doubt Apple would do that, especially considering it’s OS level integration. At best, they’d probably only have a self-report mechanism which would also likely be proprietary and therefore not verifiable in itself.


    • Malicious actors could potentially exploit vulnerabilities in the AI system to gain unauthorized access or control over device functions and data, potentially leading to severe privacy breaches, unauthorized data access, or even the ability to inject malicious content or commands through the AI system.
    • Privacy breaches are possible if the AI system is compromised, exposing user data, activities, and conversations processed by the AI.
    • Integrating AI functionality deeply into the operating system increases the overall attack surface, providing more potential entry points for malicious actors to exploit vulnerabilities and gain unauthorized access or control.
    • Human reviewers have access to annotate and process user conversations for improving the AI models. To effectively train and improve the AI models powering the OS-level integration, Apple would likely need to collect and process user data, such as text inputs, conversations, and interactions with the AI.
    • Apple’s privacy policy states that the company collects data necessary to provide and improve its products and services. The OS-level AI would fall under this category, allowing Apple to collect data processed by the AI for improving its functionality and models.
    • Despite privacy claims, Apple has a history of collecting various types of user data, including device usage, location, health data, and more, as outlined in their privacy policies.
    • If Apple partners with third-party AI providers, there is a possibility of user data being shared or accessed by those entities, as permitted by Apple’s privacy policy.
    • With the AI system operating at the OS level, it likely has access to a wide range of user data, including text inputs, conversations, and potentially other sensitive information. This raises privacy concerns about how this data is handled, stored, and potentially shared or accessed by the AI provider or other parties.
    • Lack of transparency for users about when and how their data is being processed by the AI system & users not being fully informed about data collection related to the AI. Additionally, if the AI integration is controlled solely at the OS level, users may have limited control over enabling or disabling this functionality.